Method for determining predicted value, encoder and computer storage medium

Abstract

The embodiment of the present application discloses a method for determining a prediction value, an encoder, and a computer storage medium. The method includes: obtaining an image block to be encoded and an encoded image block from a current image frame, constructing a BV candidate list for the image block to be encoded according to the encoded image block, performing an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation, selecting the BV of the image block to be encoded from the BV candidate list after the iterative operation, calling a preset affine motion model according to the BV of the image block to be encoded, calculating a reference image block of the image block to be encoded, and using the pixel value of the reference image block as the prediction value of the image block to be encoded.

Description

Technical Field

[0001] Embodiments of the present application relate to video coding technology, and more particularly to a method for determining a prediction value, an encoder, and a computer storage medium. Background Art

[0002] In the Audio Video Coding Standard (AVS), the main coding mode of Screen Content Coding (SCC) is the traditional intra-frame prediction mode (Intra mode). In Versatile Video Coding (VVC), SCC has two main coding modes, namely the traditional Intra mode and the Intra Block Copy mode (IBC mode).

[0003] However, Intra mode only predicts and encodes based on spatial texture. Compared with natural acquisition videos, the correlation of SCC video image texture is weaker, and the difference between the predicted value determined by Intra mode and the actual value is large. Although IBC mode can take advantage of the characteristics of SCC with many repeated areas, it still performs intra-frame prediction based on the translational motion model. In the translational model IBC mode, all pixels in a coding unit share the same motion vector (MV). However, in the SCC scenario, there are still complex motion situations such as scaling, rotation, deformation, and perspective transformation, and there is also a large error between the predicted value based on the translational motion model and the actual value. Summary of the invention

[0004] The embodiments of the present application provide a method for determining a prediction value, an encoder, and a computer storage medium, which can improve encoding efficiency.

[0005] The technical solution of the embodiment of the present application can be implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for determining a prediction value, the method being applied in an encoder, the method comprising:

[0007] Obtaining the image block to be encoded and the encoded image block from the current image frame;

[0008] According to the encoded image block, construct a block vector BV candidate list for the image block to be encoded;

[0009] Performing an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation;

[0010] Selecting the BV of the image block to be encoded from the BV candidate list after the iterative operation;

[0011] According to the BV of the image block to be encoded, a preset affine motion model is called to calculate a reference image block of the image block to be encoded, and a pixel value of the reference image block is used as a prediction value of the image block to be encoded.

[0012] In a second aspect, an embodiment of the present application provides an encoder, the encoder comprising:

[0013] An acquisition module, used for acquiring the image blocks to be encoded and the encoded image blocks from the current image frame;

[0014] A construction module, configured to construct a block vector BV candidate list for the image block to be encoded according to the encoded image block;

[0015] An iteration module, used for performing an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation;

[0016] A selection module, configured to select the BV of the image block to be encoded from the BV candidate list after the iterative operation;

[0017] The determination module is used to call a preset affine motion model according to the BV of the image block to be encoded, calculate a reference image block of the image block to be encoded, and use the pixel value of the reference image block as the prediction value of the image block to be encoded.

[0018] In a third aspect, an embodiment of the present application provides an encoder, the encoder comprising:

[0019] A processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the method for determining the prediction value described in the first aspect is executed.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by one or more processors, the processors execute the method for determining the prediction value described in the first aspect.

[0021] The embodiment of the present application provides a method for determining a prediction value, an encoder, and a computer storage medium. The encoder obtains an image block to be encoded and an encoded image block from a current image frame, constructs a BV candidate list for the image block to be encoded according to the encoded image block, performs an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm, obtains a BV candidate list after the iterative operation, selects the BV of the image block to be encoded from the BV candidate list after the iterative operation, calls a preset affine motion model according to the BV of the image block to be encoded, calculates a reference image block of the image block to be encoded, and converts the BV of the image block to a reference image block. The pixel value of the reference image block is used as the prediction value of the image block to be encoded; that is, in the embodiment of the present application, by constructing a BV candidate list for the image block to be encoded, and iteratively calculating each BV in the BV candidate list, the BV of the image block to be encoded can be selected from the BV candidate list after the iterative calculation, so that a more accurate BV can be determined for the image block to be encoded, and then the affine motion model is called to predict the prediction value by using affine motion compensation during intra-frame prediction encoding. In this way, the determined prediction value is closer to the true value, which effectively improves the encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A This is a schematic diagram of the arrangement of coding units when the prediction splitting mode is "NO_SPLIT";

[0023] Figure 1B It is a schematic diagram of the arrangement of coding units with a prediction partitioning mode of "HOR_tN";

[0024] Figure 1C It is a schematic diagram of the arrangement of coding units with the prediction partitioning mode of "HOR_UP";

[0025] Figure 1D It is a schematic diagram of the arrangement of coding units with the prediction partitioning mode of "HOR_DOWN";

[0026] Figure 1E It is a schematic diagram of the arrangement of coding units with a prediction partitioning mode of "VER_tN";

[0027] Figure 1F This is a schematic diagram of the arrangement of coding units with a prediction partitioning mode of "VER_LEFT";

[0028] Figure 1G It is a schematic diagram of the arrangement of coding units with the prediction partitioning mode of "VER_RIGHT";

[0029] Figure 2 Schematic diagram of intra prediction mode in AVS3;

[0030] Figure 3AIt is a schematic diagram of the structure of the coding unit after QT division;

[0031] Figure 3B It is a schematic diagram of the structure of the coding unit after vertical BT division;

[0032] Figure 3C It is a schematic diagram of the structure of the coding unit after horizontal BT division;

[0033] Figure 3D It is a schematic diagram of the structure of the coding unit after vertical TT division;

[0034] Figure 3E It is a schematic diagram of the structure of the coding unit after horizontal TT division;

[0035] Figure 4 This is a schematic diagram of the arrangement of 67 prediction directions;

[0036] Figure 5 This is a schematic diagram of the arrangement of the mapping relationship of BV in IBC Mode;

[0037] Fig. 6A Schematic diagram of the arrangement of image blocks to be encoded using four-parameter Affine;

[0038] Figure 6B Schematic diagram of the structure of the image block to be encoded using six-parameter Affine;

[0039] Figure 7 Schematic diagram of arrangement of MVs of sub-blocks obtained based on affine transformation prediction method of sub-blocks;

[0040] Figure 8 It is a schematic diagram of the test result of the first frame image of the SCC test sequence on the instrument;

[0041] Fig.9A is a schematic diagram of mapping between coding units and reference units as a scaling model;

[0042] Fig. 9B is a schematic diagram of mapping between the coding unit and the reference unit as a translation model;

[0043] Fig.10 A flowchart of an optional method for determining a prediction value provided in an embodiment of the present application;

[0044] Fig.11A A schematic diagram of blind search in a preset area using four-parameter Affine;

[0045] Fig. 11B A schematic diagram of blind search in a preset area using six-parameter Affine;

[0046] Fig.12A schematic diagram of an optional arrangement of image blocks to be encoded provided in an embodiment of the present application;

[0047] Fig.13A The diagram is a schematic diagram of the BV arrangement of the translation relationship of the four-parameter Affine;

[0048] Fig. 13B It is a schematic diagram of the arrangement of BV of the non-translation relationship of the four-parameter Affine;

[0049] Fig. 13C The diagram is a schematic diagram of the BV arrangement of the translation relationship of the six-parameter Affine;

[0050] Fig.13D The schematic diagram of the arrangement of BV of the non-translation relationship of the six-parameter Affine;

[0051] Fig.13E It is a schematic diagram of the arrangement of translation relations in SCC;

[0052] Fig.13F Schematic diagram of the arrangement of non-translational relations in SCC;

[0053] Fig.14 A schematic diagram of another optional arrangement of image blocks to be encoded provided in an embodiment of the present application;

[0054] Fig.15 A schematic diagram of the arrangement of the first optional BV candidate list provided in the embodiment of the present application;

[0055] Fig.16A A schematic diagram of a second optional arrangement of constructing a BV candidate list provided in an embodiment of the present application;

[0056] Fig. 16B A schematic diagram of a third optional arrangement of constructing a BV candidate list provided in an embodiment of the present application;

[0057] Fig.17A A schematic diagram of a third optional arrangement of constructing a BV candidate list provided in an embodiment of the present application;

[0058] Fig. 17B A schematic diagram of a fourth optional arrangement of constructing a BV candidate list provided in an embodiment of the present application;

[0059] Fig.18A A fifth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application;

[0060] Fig.18B A sixth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application;

[0061] Fig.19AA seventh optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application;

[0062] Fig.19B An eighth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application;

[0063] Fig.19C A ninth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application;

[0064] Fig.19D A schematic diagram of the arrangement of the tenth optional BV candidate list provided in the embodiment of the present application;

[0065] Fig. 20 A schematic diagram of an optional process of constructing a BV candidate list provided in an embodiment of the present application;

[0066] Fig.21 A schematic diagram of an optional process flow using an iterative calculation method provided in an embodiment of the present application;

[0067] Fig.22A A schematic diagram of an optional process for determining a coding mode provided in an embodiment of the present application;

[0068] Fig. 22B A schematic diagram of another optional flow chart for determining a coding mode provided in an embodiment of the present application;

[0069] Fig.23 A schematic flow chart of a method for inferring an optional coding mode provided in an embodiment of the present application;

[0070] Fig.24 A schematic flow chart of another optional method for inferring a coding mode provided in an embodiment of the present application;

[0071] Fig.25 A schematic diagram of the structure of an optional encoder provided in an embodiment of the present application;

[0072] Fig.26 A schematic diagram of the structure of another optional encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for the convenience of description, only the parts related to the related applications are shown in the accompanying drawings.

[0074] Nowadays, in AVS3, SCC mainly uses Intra Mode, which performs predictive coding based on spatial texture. Taking the brightness component value of the image as an example, the coding unit type and related information when IntraCuFlag is 1 are shown in Table 1 below:

[0075] Table 1

[0076] CuType SplitMode NumOfIntraPredBlock I_2Mx2N I_NO_SPLIT 1 I2M HOR_tN 4 I2T HOR_UP 2 I2M HOR_DOWN 2 I_hM_2 VER_tN 4 I_nL_2 VER_LEFT 2 I_nR VER_RIGHT 2

[0077] As shown in Table 1, if the IntraCuFlag of the current coding unit is 1, the prediction division mode (SplitMode) and the number of intra-frame luminance prediction blocks (NumOfIntraPredBlock) can be determined by looking up Table 1 according to the coding unit type (CuType). The prediction division mode (SplitMode) indicates the way in which the coding unit is divided into intra-frame prediction blocks.

[0078] Figure 1A This is a schematic diagram of the arrangement of coding units with the prediction partitioning mode being "NO_SPLIT", such as Figure 1A , the coding unit 2M×2N is not divided and is a whole block; Figure 1B Schematic diagram of the arrangement of the coding unit with the prediction partitioning mode of "HOR_tN", the encoder unit 2M×2N is divided into 0, 1, 2, 3, a total of 4 blocks, each of which is 2M×0.5N in size; Figure 1C Schematic diagram of the arrangement of the coding unit with the prediction partitioning mode of "HOR_UP". The encoder unit 2M×2N is divided into 0 and 1, a total of 2 blocks, and the size of each block is 2M×0.5N and 2M×1.5N respectively; Figure 1D Schematic diagram of the arrangement of the coding unit with the prediction partitioning mode of "HOR_DOWN". The encoder unit 2M×2N is divided into 0 and 1, a total of 2 blocks, with sizes of 2M×1.5N and 2M×0.5N respectively; Figure 1E Schematic diagram of the arrangement of the coding unit with the prediction partitioning mode of "VER_tN", the encoder unit 2M×2N is divided into 0, 1, 2, 3, a total of 4 blocks, each block size is 0.5M×2N; Figure 1F Schematic diagram of the arrangement of coding units with the prediction partitioning mode of "VER_LEFT". The coding unit 2M×2N is divided into 0 and 1, a total of 2 blocks, each of which is 0.5M×2N and 1.5M×2N respectively; Figure 1G This is a schematic diagram of the arrangement of coding units with a prediction partitioning mode of "VER_RIGHT". The coding unit 2M×2N is divided into 0 and 1, a total of 2 blocks, and the sizes of each block are 1.5M×2N and 0.5M×2N respectively.

[0079] In AVS3, the intra-frame prediction mode can be obtained according to the value of IntraLumaPredMode, as shown in Table 2 below:

[0080] Table 2

[0081] - IntraLumaPredMode Intra prediction mode 0 Intra_Luma_DC 1 Intra_Luma_Plane 2 Intra_Luma_Bilinear 3～11 Intra_Luma_Angular 12 Intra_Luma_Vertical 13～23 Intra_Luma_Angular 24 Intra_Luma_Horizontal 25～32 Intra_Luma_Angular 33 Intra_Luma_PCM

[0082] Figure 2 A schematic diagram of the intra-frame prediction mode in AVS3. Figure 2 The different prediction modes shown can calculate the prediction error between the predicted value and the true value of the image block to be encoded under each prediction mode. Among them, the prediction mode with the smallest coding overhead (error and redundant information) is the prediction mode that best matches the image block to be encoded.

[0083] In addition, in VVC, there are two main coding modes for SCC, namely the traditional Intra Mode and IBC Mode. Among them, IBC Mode is only turned on when encoding SCC sequences and Class F sequences, while Intra Mode is applicable to the used sequences.

[0084] VVC uses quadtree (QT, Quad Tree) / ternary tree (TT) / binary tree (QT, Binary Tree) for coding tree unit (CTU). Figure 3A It is a schematic diagram of the structure of the coding unit after QT division, such as Figure 3A As shown, VVC uses QT to divide CTU into 4 sub-blocks; Figure 3B It is a schematic diagram of the structure of the coding unit after vertical BT division, such as Figure 3B As shown, VVC uses vertical BT to divide CTU into 2 sub-blocks; Figure 3C It is a schematic diagram of the structure of the coding unit after horizontal BT division, such as Figure 3C As shown, VVC uses horizontal BT to divide CTU into 2 sub-blocks; Figure 3D It is a schematic diagram of the structure of the coding unit after vertical TT division, such as Figure 3D As shown, VVC divides CTU into three sub-blocks using vertical TT; Figure 3E It is a schematic diagram of the structure of the coding unit after horizontal TT division, such as Figure 3E As shown, VVC divides CTU into three sub-blocks using horizontal TT; in this way, CTU is divided into non-overlapping small blocks as the basic unit of coding, and then encoded.

[0085] Figure 4 This is a schematic diagram of the arrangement of 67 prediction directions, such as Figure 4As shown in the figure, VVC expands the number of intra-frame prediction mode angles to 65 to better adapt to textures in different directions. Together with the two prediction directions of planar and DC, there are a total of 67 optional prediction directions.

[0086] based on Figure 4 , the prediction error between the predicted block and the true value in each prediction direction can be calculated, among which the prediction direction with the smallest rate-distortion cost (Rdcost, Rate-distortion Cost) (equivalent to the coding overhead) is the prediction direction that best matches the block.

[0087] For IBC Mode, IBC Mode is only turned on when encoding SCC sequences and Class F sequences. IBC Mode is similar to the traditional inter-frame mode. It uses motion vectors (MV) to establish a reference relationship between the image block to be encoded and the reference image block, so as to predict the pixels of the image block to be encoded. The difference is that in IBC Mode, the image block to be encoded and the reference image block are in the same frame. In order to distinguish it from the inter-frame mode, block vectors (BV, BlockVector) are used to represent the reference relationship in IBC Mode.

[0088] Figure 5 The following is a schematic diagram of the mapping relationship between BVs in IBC Mode. Figure 5 As shown, IBC Mode performs motion estimation based on the translation motion model. All pixels in the coding prediction unit (PU) share the same BV. Limited by the prediction effect of the translation model, IBC Mode is only applicable to coding units whose width and height are less than or equal to 64 pixels (for large PUs, all pixels in the PU share one BV, which will cause a large prediction error, increase the encoding time and have poor effect, so IBC Mode is limited to small blocks).

[0089] In AVS3 and VVC, a new technology is used for inter-frame prediction: affine motion model Affine. When the configuration file allows the use of Affine affine motion, it means that the encoder can use affine motion compensation. Affine is a high-order motion model that can better describe the complex motions of the image blocks to be encoded, such as scaling, rotation, deformation, perspective, and irregular angle rotation. Among them, Affine is divided into four-parameter and six-parameter types.

[0090] Fig. 6A is a schematic diagram of the arrangement of the image blocks to be encoded using the four-parameter Affine, as shown in Fig. 6A As shown, we only need to get the MV (v0, v1) of the two control points in the upper left corner and the upper right corner. Figure 6Bis a schematic diagram of the structure of the image block to be encoded using six-parameter Affine, such as Figure 6B As shown in the figure, we need to obtain the MVs (v0, v1, v2) of the three control points at the upper left corner, upper right corner, and lower left corner. Then we can infer the MV (v0, v1, v2) of each pixel in the image block w×h to be encoded according to the following formula (1) or the following formula (2): x , v y ), so when the MV of each pixel is different, a non-translational effect is achieved, and when the MV of each pixel is the same, it is equivalent to a translation model.

[0091]

[0092] Among them, in the four-parameter Affine, the coordinates of the control point v0 in the upper left corner are (v0x, v0y), and the coordinates of the control point v1 in the upper right corner are (v1x, v1y). In the six-parameter Affine, compared with the four-parameter Affine, the coordinates of the newly added control point v1 in the lower left corner are (v2x, v2y).

[0093] In addition, AVS3 uses a sub-block-based affine transformation prediction method. Figure 7 Schematic diagram of the arrangement of the MV of the sub-block obtained by the affine transformation prediction method based on the sub-block, such as Figure 7 As shown in FIG. 1 , if the image block to be encoded is in affine mode (AffineFlag is 1), each sub-block of equal size in the image block to be encoded shares one MV (calculated according to formula (1) or (2)), instead of calculating one MV for each pixel point. This not only achieves the effect of non-translation, but also effectively reduces the complexity of encoding. The size of the sub-block can be controlled by AffineSubblockSizeFlag. When AffineSubblockSizeFlag is 0, the sub-block size is set to 4×4, otherwise it is set to 8×8.

[0094] In order to simplify motion compensation prediction, VVC also uses a sub-block-based affine transformation prediction method. That is, each 4×4 sub-block in the image block to be encoded shares one MV, rather than calculating an MV for each pixel separately. This not only achieves the effect of non-translation, but also effectively reduces the complexity of encoding.

[0095] That is to say, in AVS3 and VVC, the traditional Intra Mode only predicts based on spatial texture, and does not take advantage of the fact that compared with naturally acquired videos, the SCC video image texture has weaker correlation but more repeated or similar areas.

[0096] Moreover, in VVC, although IBC Mode can utilize the characteristics of many repeated areas, it still performs intra-frame prediction based on the translational motion model. In the translational model IBC Mode, all pixels in a coding unit share the same MV, but in the SCC scenario, there are still complex motions such as scaling, rotation, deformation, and perspective transformation. The prediction results based on the translational motion model have large errors. In addition, when there are non-translational complex motions such as rotation or scaling, in order to meet the image quality, in the IBC Mode based on translational motion, the encoder tends to divide an object into very small units, and use the translational motion of small units to approximate the complex motion. This division and prediction method will bring a lot of redundant information, for example, such as division information, which will affect the compression performance.

[0097] Figure 8 This is a schematic diagram of the test results of the first frame image of the SCC test sequence on the instrument, as shown in Figure 8 As shown, the translation motion model enables the current coding unit to search only for units with the same size as itself and without complex motions such as rotation angle and deformation as its reference within the area where the coding has been completed.

[0098] Fig.9A is a schematic diagram of the mapping between the coding unit and the reference unit as a scaling model, such as Fig.9A As shown, for any two unequal rectangles A and B, the ideal reference process is as follows Fig.9A As shown, rectangle B can be directly encoded with A as the reference unit through a non-translation model (including scaling motion). However, for a translation motion model, Fig. 9B The mapping diagram of the coding unit and the reference unit is a translation model. The actual reference process is as follows Fig. 9B As shown, a larger coding unit is divided into units B, C and D of the same size as A, so that units B, C and D can be encoded respectively with A as the mapping reference; thereby introducing a large amount of redundant division information and motion parameters, which greatly affects the coding performance.

[0099] In addition, for example, in stock market charts, lines may rise, fall, and oscillate, and repeated areas are less likely to appear in the same frame. In this case, the translational motion model of IBC Mode does not work well.

[0100] In order to make the predicted value of the image block to be encoded closer to the actual value of the image block to be encoded, the embodiment of the present application provides a method for determining the predicted value. Fig.10 A flowchart of an optional method for determining a prediction value provided in an embodiment of the present application is shown as follows: Fig.10 As shown, the method is applied to an encoder, and the method may include:

[0101] S1001: Obtaining an image block to be encoded and an encoded image block from a current image frame;

[0102] Here, for AVS3 and VVC, Affine is introduced on the basis of intra-frame prediction, and a new coding mode is provided. In AVS3, it is called the intra-frame affine motion coding mode Intra Affine Mode, and in VVC, it is called the current frame reference (CPR, Current Picture Referencing)-intra-frame affine motion coding mode Intra AffineCPR Mode.

[0103] Among them, Intra Affine Mode and Intra Affine CPR Mode are newly added coding modes in addition to the original Intra Mode. They are independent of the original Intra Mode, do not interfere with each other, and compete with each other. Ultimately, the Rdcost is calculated by the coding process, and the optimal coding mode is determined based on the Rdcost.

[0104] In the new encoding mode, when encoding the current image frame, the encoder first obtains the image block to be encoded and the encoded image block, and then determines the prediction value of the image block to be encoded according to the pixel value of the encoded image block.

[0105] Among them, the type of the above-mentioned pixel value can be the pixel value of the brightness component of the image block, or it can be the pixel value of the chrominance component of the image block. Here, the embodiment of the present application does not make any specific limitation on this.

[0106] S1002: constructing a BV candidate list for the image block to be encoded according to the encoded image block;

[0107] In order to determine the prediction value of the image block to be encoded, firstly, a BV candidate list needs to be constructed for the image block to be encoded.

[0108] Specifically, before constructing the BV candidate list, the image block to be encoded needs to be screened to determine whether the image block to be encoded is suitable for Intra Affine Mode or Intra Affine CPR Mode; in an optional embodiment, S1002 may include:

[0109] Determine whether the length and width of the image block to be encoded are both greater than or equal to the distance of 16 pixels;

[0110] When the length and width of the image block to be encoded are both greater than or equal to the distance of 16 pixels, a BV candidate list is constructed for the image block to be encoded based on the encoded image block;

[0111] When the length or width of the image block to be encoded is less than the distance of 16 pixels, the image block to be encoded is encoded in intra-frame mode to obtain a predicted value of the image block to be encoded.

[0112] Here, after obtaining the image block to be encoded, it is determined whether the length of the image block to be encoded is greater than or equal to the distance of 16 pixels, and whether the width of the image block to be encoded is greater than or equal to the distance of 16 pixels. Only when the length of the image block to be encoded is greater than or equal to the distance of 16 pixels and the width of the image block to be encoded is greater than or equal to the distance of 16 pixels, Intra Affine Mode or Intra Affine CPR Mode is used for encoding, that is, a BV candidate list is constructed for the image block to be encoded based on the encoded image block, otherwise, the traditional encoding mode is used for encoding.

[0113] Here, it should be noted that the reason why the Intra-Affine Mode proposed in the embodiment of the present application selects image blocks whose length and width are both greater than or equal to the distance of 16 pixels to be encoded is as follows:

[0114] Compared with the translation model, Affine can better predict large blocks and reduce prediction errors. In addition, the redundant information brought by the coding unit will be greatly reduced, such as division information and motion parameters, which is conducive to bit rate reduction. In order to reduce complexity, coding units with a length and / or width less than 16 will skip the Intra-Affine Mode. In addition, in Affine, each sub-block of equal size will have an independent BV that can be derived and does not need to be transmitted, which can achieve sufficient prediction accuracy, and there is little point in continuing to divide.

[0115] In SCC, the main coding mode of AVS3 is the traditional Intra Mode. In an optional embodiment, when the coding mode adopted by the encoder is the intra mode, S1002 may include:

[0116] Acquire a preset area adjacent to the image block to be encoded;

[0117] According to the starting point of the control point set for the preset area and the preset search step length, the preset area is searched to find the control point for the image block to be encoded;

[0118] Calculate the searched BV for the image block to be encoded based on the searched control points and the control points of the image block to be encoded;

[0119] The searched BV is added to the BV candidate list.

[0120] Specifically, in constructing the BV candidate list, the initial BV candidate list can be established by reusing the motion parameter information of the encoded spatial adjacent image blocks, but the premise is that there is at least one encoded image block in the adjacent image blocks whose coding mode is Intra-Affine Mode; then, when the Intra-Affine Mode is used to construct the BV candidate list, it is necessary to continuously inherit the motion parameters of the encoded area of ​​the current frame, which leads to the following problems: the first coding unit to enter the IntraAffine Mode cannot construct the BV candidate list through the encoded spatial adjacent image blocks, which makes it impossible to use the Intra Affine Mode; in addition, the BV candidate list constructed only through adjacent image blocks may not be enough, which is not conducive to finding the optimal BV; therefore, the embodiment of the present application proposes a blind search method, which can perform a blind search in the largest coding unit (LCU, Largest Coding Unit) where the current coding unit is located, and further expand the BV candidate list of the image block to be encoded.

[0121] It should be noted that, for AVS3, during the encoding process, the first image block to be encoded using the Intra Affine Mode must use a blind search method to construct a BV candidate list.

[0122] For example, Fig.11A This is a schematic diagram of using four-parameter Affine to perform blind search in a preset area. Fig. 11B This is a schematic diagram of using six-parameter Affine to perform blind search in a preset area. Below, taking four-parameter Affine as an example, the blind search method is as follows:

[0123] A coordinate system is established with the first pixel point in the upper left corner of the LCU where the current coding unit (equivalent to the above-mentioned image block to be encoded) is located as the coordinate origin, the width direction is the horizontal coordinate, and the height direction is the vertical coordinate. The current LCU size is W×H, the current coding unit size is w×h, the pixel position of the upper left corner control point (LT) of the current coding unit is (x1, y1), the pixel position of the upper right corner control point (RT) is (x2, y2), and (x2, y2)=(x1+w, y1).

[0124] like Fig.11A As shown, the preset area consists of an encoded area A and an encoded area B, wherein the horizontal coordinate range of area A is [0, W-1] and the vertical coordinate range is [0, y-1], and the horizontal coordinate range of area B is [0, x-1] and the vertical coordinate range is [y, y+H-1].

[0125] The reference points of the control points LT and RT of the current coding unit are searched in regions A and B respectively, which are represented as Ref_LT(x 01 ,y 01 ) and Ref_RT(x 02 ,y 02 ), calculate the BV (v) of the LT control point according to the coordinates of LT and Ref_LT LT ), and similarly, we get the BV(v RT ), the BVs of the two control points form a BV (v LT , v RT ), as the search starting point for motion estimation, as follows:

[0126]

[0127] Search for Ref_LT(x 01 ,y 01 ) and Ref_RT(x 02 ,y 02 ), since the current coding unit is at least 16×16, taking into account the time complexity of the search, the search step can be set according to the needs, wherein the smaller the search step, the finer the prediction, but more time-consuming; here, the embodiment of the present application does not specifically limit the value of the search step.

[0128] Taking the search step length of 8 pixels as an example, the specific steps of the search process are as follows:

[0129] First, set the search starting point of Ref_LT, that is, the pixel point at the upper left corner of area A or B where Ref_LT is located;

[0130] Then, the position of Ref_LT is fixed to search for Ref_RT, that is, Ref_RT is shifted rightward or downward according to the search step of 8 pixels. When Ref_LT and Ref_RT are in the same area, the search for Ref_RT starts from the pixel position of Ref_LT; otherwise, the search starts from the pixel position of the upper left corner of the area A or B where Ref_RT is located. During the search, Ref_RT is restricted to not appear on the left side of Ref_LT.

[0131] When Ref_RT completes the search in the area where it is located, Ref_LT is updated, that is, Ref_LT is translated rightward or downward according to the search step of 8 pixels, and the Ref_RT corresponding to the current Ref_LT is obtained.

[0132] The blind search ends when Ref_LT completes the search in its area A and B;

[0133] It should be noted that Fig. 11BThe blind search of the six-parameter Affine in the preset area is similar to the blind search of the four-parameter Affine mentioned above, and will not be repeated here.

[0134] For AVS3, in an optional embodiment, S1002 may include:

[0135] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0136] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0137] The BV candidate list of adjacent image blocks is constructed from the searched BVs.

[0138] Specifically, Fig.12 A schematic diagram of an optional arrangement of image blocks to be encoded provided in an embodiment of the present application, such as Fig.12 As shown, in the encoded image block, the adjacent image blocks of the image block to be encoded include A, B, C, D, G and F. If the encoding mode of the adjacent image block adopts Intra Affine Mode, its BV is directly reused and filled into the BV candidate list.

[0139] For example, the BV of the adjacent image block is obtained through blind search, which means that the encoding mode adopted by the adjacent image block is Intra Affine Mode, so it is added to the BV candidate list.

[0140] In an optional embodiment, when the Intra Affine Mode is used for adjacent image blocks, after selecting adjacent image blocks of the image block to be encoded from the encoded image blocks, the method may further include:

[0141] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0142] The BV candidate list of adjacent image blocks is constructed from the searched BV and / or the BV candidate list of adjacent image blocks of the adjacent image blocks.

[0143] Here, if the encoding mode of adjacent image blocks adopts Intra Affine Mode, then the BV candidate list of the adjacent image blocks is the searched BV and / or the BV candidate list of adjacent image blocks of the adjacent image blocks. That is to say, when the adjacent image blocks adopt Intra Affine Mode, the adjacent image blocks are the BV candidate list obtained by blind search and / or the BV candidate list obtained by the BV candidate list of adjacent image blocks of the adjacent image blocks. Therefore, the BV candidate list of the adjacent image blocks is added to the BV candidate list.

[0144] In the embodiment of the present application, taking the SCC of AVS3 as an example, in the Intra Affine Mode, for the four-parameter Affine, Fig.13A This is a schematic diagram of the BV arrangement of the translation relationship of the four-parameter Affine, such as Fig.13A As shown, if the BV of the upper left corner control point (LT) and the upper right corner control point (RT) of the current coding block are equal, it means that the reference relationship is a translation relationship; Fig. 13B This is a schematic diagram of the arrangement of BV of the non-translation relationship of the four-parameter Affine, such as Fig. 13B As shown, if they are not equal, it means that it is a non-translation complex motion reference relationship such as scaling and rotation; similarly, for the six-parameter Affine, Fig. 13C This is a schematic diagram of the BV arrangement of the translation relationship of the six-parameter Affine, such as Fig. 13C As shown, if the BV of the current coding block LT, RT and the lower left corner control point (LB) are equal, it means that the reference relationship is a translation relationship. Fig.13D This is a schematic diagram of the arrangement of BV of the non-translation relationship of the six-parameter Affine, such as Fig.13D As shown, if they are not equal, it means that it is a non-translational complex motion reference relationship such as scaling and rotation.

[0145] In the case of a translation reference relationship, the characteristic of the SCC sequence with many repeated regions can bring about bit rate savings. In this case, the BVs of the sub-blocks derived according to formula (1) or (2) in a coding unit are exactly the same. Fig.13E This is a schematic diagram of the arrangement of translation relationships in SCC, such as Fig.13E As shown, the current coding unit will refer to a coding unit that has the same width and height as the current coding unit and is located in the encoded area of ​​the current frame.

[0146] When there are complex motions such as scaling, rotation, deformation, and perspective transformation, the prediction result of translation motion has a large error. However, Affine is a high-order motion model and can exist in a non-translation reference relationship. The characteristic of many similar regions in the SCC sequence under the non-translation reference relationship can bring bit rate savings. At this time, the BVs of the sub-blocks derived according to formula (1) or (2) in a coding unit are different from each other to simulate complex motions such as scaling, rotation, and perspective. Fig.13F It is a schematic diagram of the arrangement of non-translation relations in SCC, such as Fig.13F As shown, the current coding unit will refer to a coding unit that is different in width and height and is located in the coded area of ​​the current frame (the BV of each sub-block is different, and the referenced sub-blocks may not be located in the same coding unit).

[0147] In this way, the non-translational reference relationship for SCC is realized through the Intra Affine Mode, which can improve the accuracy of intra-frame prediction and improve coding efficiency. This technology can be used in common SCC scenarios such as game live broadcast, remote desktop, online games, etc., and has good application prospects.

[0148] Similarly, for VVC's SCC, Intra Affine CPR Mode can also achieve non-translational reference relationship, improve the accuracy of intra-frame prediction, and improve coding efficiency. This technology can be used in common SCC scenarios such as game live broadcast, remote desktop, online games, etc., and has good application prospects.

[0149] For VVC, the main coding modes adopted by SCC are Intra Mode and IBC Mode. In an optional embodiment, when the coding mode adopted by the encoder is the intra mode and the intra block copy IBC mode, S1002 may include:

[0150] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0151] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0152] The BV candidate list of the adjacent image blocks is constructed by the BV of the reference image blocks of the adjacent image blocks.

[0153] Specifically, since the IBC Mode coding mode can be used in VVC, if the coding mode adopted by the BV of the adjacent image block is IBC Mode, that is, when the BV candidate list of the adjacent image block is constructed by the BV of the reference image block of the adjacent image block, the BV candidate list of the adjacent image block is added to the BV candidate list, and the mode of encoding the BV candidate list constructed in this way is called Intra Affine CPR Mode.

[0154] Fig.14 Another optional arrangement diagram of the image blocks to be encoded provided in the embodiment of the present application is as follows: Fig.14 As shown, the BV candidate list can be obtained from the BV of the encoded neighboring PU of CurPU. The access order of the neighboring PU is from the left neighboring PU (A0) to the lower left neighboring PU (A1), to the upper neighboring PU (B0), to the upper right neighboring PU (B1) and then to the upper left neighboring PU (B2).

[0155] Fig.15 The first optional arrangement diagram of constructing the BV candidate list provided in the embodiment of the present application is as follows: Fig.15As shown in the figure, the four-parameter Affine is used. When the coding mode adopted by the adjacent PU is IBC Mode, the BV candidate list of the adjacent image block is copied to all control points of the current PU; although the initial BV candidate list of the control point is the same, the subsequent iterative update will adjust the BV candidate list to achieve the effect of the non-translational model.

[0156] Fig.16A The second optional arrangement diagram of constructing the BV candidate list provided in the embodiment of the present application adopts the four-parameter Affine. If the adjacent PU is Intra Affine CPR Mode, the BV candidate list of the adjacent PU can be directly reused into the BV candidate list, such as Fig.16A shown.

[0157] Fig. 16B The third optional arrangement diagram of constructing the BV candidate list provided in the embodiment of the present application adopts the six-parameter Affine. If the adjacent PU is Intra Affine CPR Mode, the BV candidate list of the adjacent PU can be directly reused into the BV candidate list, such as Fig. 16B shown.

[0158] In order to construct a BV candidate list for an image block to be encoded in VVC, in an optional embodiment, S102 may include:

[0159] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0160] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0161] The BV candidate list of adjacent image blocks is constructed from the BV candidate lists of adjacent image blocks of adjacent image blocks.

[0162] Here, the BV candidate list of the selected adjacent image blocks is constructed by the BV candidate list of the adjacent image blocks of the adjacent image blocks, indicating that the adjacent image blocks of the adjacent image blocks adopt the IBC Mode or the IntraAffine CPR Mode coding mode, so the BV candidate list of the adjacent image blocks is added to the BV candidate list.

[0163] For VVC, since SCC can adopt IBC Mode, the BV candidate list of the adjacent image block adopting IBC Mode can be directly reused. Therefore, the blind search method is an optional method for constructing the BV candidate list in VVC, and is not a necessary method for constructing the BV candidate list. In an optional embodiment, S1002 may include:

[0164] Acquire a preset area adjacent to the image block to be encoded;

[0165] According to the starting point of the control point set for the preset area and the preset search step length, the preset area is searched to find the control point for the image block to be encoded;

[0166] Calculate the searched BV for the image block to be encoded based on the searched control points and the control points of the image block to be encoded;

[0167] The searched BV is added to the BV candidate list.

[0168] Here, it should be noted that the blind search method in VVC is the same as the blind search method in AVS3, and will not be repeated here.

[0169] Among them, blind search is an optional way to construct a BV candidate list in VVC, while blind search is a necessary way to construct a candidate list in AVS3.

[0170] In order to construct a BV candidate list for an image block to be encoded in VVC, in an optional embodiment, S1002 may include:

[0171] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0172] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0173] The BV candidate list of the adjacent image blocks is constructed by the BV of the reference image block of the adjacent image block, and / or the BV candidate list of the adjacent image block of the adjacent image block, and / or the searched BV.

[0174] Here, if the encoding mode of the adjacent image blocks adopts the Intra Affine CPR Mode or IBC Mode, then the BV candidate list of the adjacent image blocks is the searched BV, and / or the BV candidate list of the adjacent image blocks of the adjacent image blocks, and / or the BV of the reference image blocks of the adjacent image blocks, that is, when the adjacent image blocks adopt the IntraAffine Mode or IBC Mode, the BV candidate list of the adjacent image blocks can be added to the BV candidate list.

[0175] In order to construct a BV candidate list for an image block to be encoded in VVC, in an optional embodiment, S1002 may include:

[0176] According to the BV of the adjacent image block, the control point of the reference image block of the adjacent image block is calculated;

[0177] The BV of the image block to be encoded and the reference image block of the adjacent image block is calculated according to the control point of the reference image block of the adjacent image block and the control point of the image block to be encoded;

[0178] The BVs of the image block to be encoded and the reference image blocks of the adjacent image blocks are added to the BV candidate list.

[0179] Specifically, when the adjacent image blocks adopt the Intra Affine CPR Mode coding mode, the BV of the adjacent image blocks is obtained first. The BV of the adjacent image blocks is the vector from the control point of the adjacent image blocks to the control point of the reference image block of the adjacent image blocks. Based on this, on the basis of knowing the BV of the adjacent image blocks, the control point of the reference image block of the adjacent image blocks can be calculated. Based on this, the vector pointing from the control point of the image block to be encoded to the control point of the reference image block of the adjacent image blocks is calculated, and the BV of the image block to be encoded and the reference image block of the adjacent image blocks is obtained, and it is added to the BV candidate list.

[0180] For example, Fig.17A A third optional arrangement diagram of constructing a BV candidate list provided in the embodiment of the present application, taking the four-parameter Affine as an example, such as Fig.17A As shown, if the adjacent image block is Intra Affine CPRMode, the two control points of the upper left and upper right corners of the reference image block of the adjacent image block can be directly reused, and the two control points of the image block to be encoded are inferred to point to the BV of the two control points of the reference image block of the adjacent image block according to the spatial position, and the inferred BV is added to the BV candidate list; taking the six-parameter Affine as an example, Fig. 17B A fourth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application is shown in FIG. Fig. 17B As shown, if the adjacent image block is IntraAffine CPR Mode, the control points of the upper left corner, upper right corner and lower left corner of the reference image block of the adjacent image block can be directly reused, and the BV of the three control points of the image block to be encoded pointing to the three control points of the reference image block of the adjacent image block can be inferred according to the spatial position, and the inferred BV is added to the BV candidate list.

[0181] In addition, when the adjacent image blocks adopt the IBC Mode coding mode, the BV of the adjacent image blocks is obtained first. The BV of the adjacent image blocks is the vector from the upper left corner control point of the image block to be encoded to the upper left corner control point of the reference image block of the adjacent image block. Based on this, on the basis of knowing the BV of the adjacent image blocks, the control point of the reference image block of the adjacent image blocks can be calculated. Based on this, the vector pointing from the control point of the image block to be encoded to the control point of the reference image block of the adjacent image block is calculated, and the BV of the image block to be encoded and the reference image block of the adjacent image block is obtained, and it is added to the BV candidate list.

[0182] For example, Fig.18A A fifth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application is shown in FIG. Fig.18A As shown, when the adjacent image blocks ( Fig.18A When the encoding mode of NeiPU in is IBC Mode, the image block to be encoded ( Fig.18A The upper left and upper right control points of CurPU in the figure point to the reference PU of NeiPU ( Fig.18A The BV of the control points of the upper left and upper right corners of NeiRefPU) is added to the BV candidate list; Fig.18B A sixth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application is shown in FIG. Fig.18B As shown, when the adjacent image blocks ( Fig.18B When the encoding mode of NeiPU in is IBC Mode, the image block to be encoded ( Fig.18B The control points of the upper left, upper right and lower left corners of the CurPU in the figure point to the reference PU of the NeiPU ( Fig.18B The BVs of the control points of the upper left, upper right and lower left corners of NeiRefPU) are added to the BV candidate list.

[0183] It should be noted that although the reference control point is the same as that of the adjacent PU, the width and height of the current PU and the adjacent PU may not be the same, so the reference sub-block pointed to by each sub-block inferred based on Affine will be different, achieving a non-translational effect.

[0184] In order to construct a BV candidate list for an image block to be encoded in VVC, in an optional embodiment, S1002 may include:

[0185] Based on the control point of the image block to be encoded, the BV of the adjacent image block is used to calculate an indirect reference image block of the image block to be encoded;

[0186] According to the BV of the adjacent image block, the control point of the indirect reference image block, the BV of the indirect reference image block and the control point of the image block to be encoded, a vector addition and subtraction algorithm is used to calculate the BV of the image block to be encoded and the indirect reference image block;

[0187] The BVs of the image block to be encoded and the indirect reference image block are added to the BV candidate list.

[0188] Specifically, after obtaining the BV of adjacent image blocks, the BV of adjacent image blocks can be reused through the image block to be encoded, the indirect reference image block of the image block to be encoded can be calculated, and the BV of the indirect reference image block can be obtained. Finally, according to the vector addition and subtraction algorithm, the BV of the image block to be encoded and the indirect reference image block is calculated and added to the BV candidate list.

[0189] For example, Fig.19A A seventh optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application is shown in FIG. Fig.19A As shown, LT represents the upper left corner control point of CurPU, RT represents the upper right corner control point of CurPU, (v0, v1) is the BV of NeiPU, when CurPU can find the indirect reference image block through the BV of NeiPU, Fig.19A PU0 and PU1 are used to represent the BV (v l1 , v r1 ) can be expressed by the following formula:

[0190]

[0191] Among them, the vector in formula (4) is as follows Fig.19A As shown, (w pu0 ,0)-(w Cur_pu ,0) represents the vector from RT to the middle point of LT, and the length of this vector is w Cur_pu -w pu0 .

[0192] Fig.19B The eighth optional arrangement diagram of constructing the BV candidate list provided in the embodiment of the present application is as follows: Fig.19B As shown, LT represents the upper left corner control point of CurPU, RT represents the upper right corner control point of CurPU, (v0, v1) is the BV of NeiPU, when CurPU can find the indirect reference image block through the BV of NeiPU, Fig.19B PU0 and PU1 are used to represent the BV (v l2 , v r2 ) is expressed by the following formula:

[0193]

[0194] Among them, the vector in formula (5) is as follows Fig.19B As shown, Fig.19B Similar, no further description is given here.

[0195] Fig.19C A ninth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application is shown in FIG. Fig.19C As shown, LT represents the upper left corner control point of CurPU, RT represents the upper right corner control point of CurPU, (v0, v1) is the BV of NeiPU, when CurPU can find the indirect reference image block through the BV of NeiPU, Fig.19C PU0 and PU1 are used to represent the BV (v l3 , v r3 ) is expressed by the following formula:

[0196]

[0197] Among them, the vector in formula (6) is as follows Fig.19C As shown, Fig.19C Similar, no further description is given here.

[0198] Among them, as above Figure 19A-Figure 19C As shown in the figure, the current PU (CurPU) first reuses the BV (v0, v1) of the spatially adjacent PU (NeiPU), finds the PU0 and PU1 pointed to by v0 and v1, and then, according to the BV (v 00 , v 01 ), PU1's BV(v 10 , v 11 ), use the above formula (4) or (5) or (6) to calculate and derive the BV of the current PU, and fill it into the BV candidate list.

[0199] Fig.19D A tenth optional arrangement diagram of constructing a BV candidate list provided in an embodiment of the present application is shown in FIG. Fig.19D As shown, LT represents the upper left corner control point of CurPU, RT represents the upper right corner control point of CurPU, (v0, v1) is the BV of NeiPU, when CurPU can find the indirect reference image block through the BV of NeiPU, Fig.19D In the figure, PU0 and PU1 are used to represent the BV obtained by using the indirect reference image block. The BV is expressed by the following formula:

[0200]

[0201] It should be noted that Figure 19A-Figure 19CThe NeiPU, PU0, and PU1 shown in the figure all use the Intra AffineCPR Mode. If one of the PUs is in IBC Mode, this indirect multiplexing method can also be used. In this case, you only need to copy its BV to all other control points, such as Fig.19D As shown, other situations are also true, which are just the combination of IBC Mode and Intra-AffineCPR Mode, and will not be repeated here.

[0202] When constructing the BV candidate list in VVC, it is necessary to continuously inherit the motion parameters of the encoded area of ​​the current frame. There are the following problems: Since the Intra Affine CPR Mode proposed last time is a new mode parallel to the IBC Mode and the traditional Intra Mode, they are completely independent, non-interfering, and competing with each other. If IBC Mode is turned off, BV cannot be reused directly or indirectly. Then, the first coding unit to enter the Intra Affine CPR Mode cannot pass the above-mentioned inherited motion estimation starting point BV, and cannot perform subsequent iterative updates to find the optimal BV, which step by step causes the subsequent coding units to be unable to inherit. In other words, the premise for the good effect of Intra Affine CPR Mode is that IBC Mode must also be turned on. In addition, the BVs found only through the above-mentioned inherited motion estimation starting point may not be enough, which is not conducive to finding the optimal BV. Therefore, a blind search method can be used in VVC.

[0203] It should be noted that there is a big difference between VVC and AVS3. AVS3 does not have IBC Mode. If there is no blind search process, it is impossible to find the starting point and it is impossible to perform subsequent iterative update steps. Therefore, AVS3 must include a blind search process. When IBC Mode is turned on, VVC can reuse the BV of IBC Mode and adjust the BV as the starting point of BV. Therefore, VVC does not necessarily include a blind search process.

[0204] For VVC, Fig. 20 A schematic diagram of an optional process for constructing a BV candidate list provided in an embodiment of the present application is as follows: Fig. 20 As shown, constructing the BV candidate list can be roughly divided into three methods: direct reuse, indirect reuse and blind search. The specific implementation steps are as follows:

[0205] S2001: Clear the initial BV candidate list, cnt_ind_inherit=0, cnt_inherit=0; execute S2002 and S2003;

[0206] S2002: Determine whether the coding mode of the spatially adjacent PU is IBC Mode or Intra Affine CPR Mode? If yes, execute S2004 and S2005; if no, end.

[0207] S2003: Construct a BV candidate list by blind search, and encode according to the constructed candidate list, and execute S2006; here, the blind search method is the same as described above, and will not be repeated here.

[0208] S2004: constructing a BV candidate list by indirect multiplexing, and performing encoding according to the constructed candidate list, and executing S2006;

[0209] S2005: constructing a BV candidate list by direct multiplexing, and performing encoding according to the constructed candidate list, and executing S2006;

[0210] S2006: Update cnt to cnt_inherit+cnt_ind_inherit+cnt_search, and end.

[0211] Among them, the specific steps of direct reuse are as mentioned above Fig.15 , Fig.16A , Fig. 16B , Fig.17A , Fig. 17B , Fig.18A and Fig.18B ; The specific steps of indirect reuse are as above FIG. 19A to FIG. 19D The blind search process is the same as the blind search process in other parts of the embodiment of the present application, and will not be repeated here.

[0212] For blind search, after a BV is searched out, the searched BV may be added to a BV candidate list, and the searched BV may be selectively added to the BV candidate list according to the coding cost of each searched BV. In an optional embodiment, after the searched BV is calculated for the image block to be encoded according to the searched control points and the control points of the image block to be encoded, the method may further include:

[0213] Calculate the first encoding cost generated by respectively encoding the image block to be encoded using each BV in the searched BV;

[0214] Selecting a preset number of BVs from the first coding overhead in descending order;

[0215] A preset number of BVs are added to the BV candidate list.

[0216] The first coding overhead is the error between the real value of the image block to be coded and the predicted value of the image block to be coded and the redundant information in the coding process.

[0217] Still Fig.11A For example, after obtaining the searched BV, for each set of Ref_LT and Ref_RT in the search process, a corresponding BV candidate list is established, and the cost of the BV is directly calculated without iterative search (the cost comprehensively considers the prediction error and other overheads, such as the introduced motion parameters), and the BVs in the searched BV candidate list are sorted according to the cost;

[0218] When all classification methods have completed the search, only 16 BVs with the smallest cost are retained as the starting points for motion estimation and added to the initial BV candidate list.

[0219] For AVS3 (or VVC), the steps of blind search of six-parameter Affine in LCU (called CTU in VVC) are similar to those of four-parameter Affine, except that the reference point Ref_LB of the lower left control point LB of the current coding unit is searched. The steps of Ref_LB search are also similar to those of Ref_RT, so they will not be repeated here.

[0220] S1003: performing an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation;

[0221] Here, an iteration formula and an iteration threshold are preset, and each BV in the BV candidate list is iteratively calculated to obtain a BV candidate list after iterative calculation. In an optional embodiment, S103 may include:

[0222] Determine whether the number of iterations i is less than or equal to a preset iteration threshold; wherein the initial value of the number of iterations i is 1;

[0223] When i is less than or equal to the iteration threshold, each BV in the BV candidate list is substituted into the error formula between the true value of the image block to be encoded and the predicted value of the image block to be encoded, and the least squares algorithm is used to update the parameters of the affine motion model in the error formula and the BV candidate list;

[0224] When the number of iterations is greater than a preset iteration threshold, the updated BV candidate list is obtained as the BV candidate list after the iterative operation, and the parameters of the affine motion model in the updated error formula are used to update the parameters in the affine motion model;

[0225] i is updated to i+1, and the process returns to execute to determine whether the number of iterations i is less than or equal to the preset number of iterations.

[0226] Specifically, using p(x i ,y i ) indicates the image block to be encoded at (x i,y i ) pixel value, p'(x i ,y i ) represents the pixel value at the corresponding position in the reference image block, (Δx, Δy) represents the displacement vector from the pixel to its reference pixel, and the predicted value of the pixel in the image block to be encoded It can be expressed as follows:

[0227]

[0228] in, and They represent the gradients of the image in the horizontal and vertical directions respectively, and can be calculated using the traditional Sobel operator.

[0229] The error formula between the true value of the image block to be encoded and the predicted value of the image block to be encoded can be expressed as:

[0230]

[0231] Where A = [aω0bω1] T are the parameters of the Affine model.

[0232] According to the least squares method, in order to minimize the prediction error, we get:

[0233]

[0234] Based on formulas (9) and (10), the Affine parameters and the BV candidate list can be updated to obtain the BV candidate list after iterative operation.

[0235] Fig.21 A schematic diagram of an optional process using an iterative calculation method provided in an embodiment of the present application is as follows: Fig.21 As shown, the maximum number of iterations of four-parameter Affine and six-parameter Affine is set to 5 and 4 respectively; taking four parameters as an example, the iterative calculation method is as follows:

[0236] S2101: Obtain each BV and iteration threshold in the BV candidate list, where the number of iterations cnt=0;

[0237] S2102: BV is respectively input into formula (9) to obtain the error formula of each BV;

[0238] S2103: Calculate the gradient matrix according to formula (10) and error d(x i ,y i) to update the parameter matrices A and BV, use the BV updated in this iteration to update the BV candidate list, and use the parameter matrix A updated in this iteration to update the parameters in Affine.

[0239] S2104: Then the number of iterations is automatically increased by 1. When the number of iterations is less than or equal to the iteration threshold, return to step S2102;

[0240] S2105: When the number of iterations is greater than the iteration threshold, the BV candidate list after the iterative operation is output.

[0241] S1004: Selecting a BV of the image block to be encoded from the BV candidate list after the iterative operation;

[0242] In order to select the best BV as the BV of the image block to be encoded, in an optional embodiment, S1004 may include:

[0243] Calculate the second coding overhead generated by respectively coding the image block to be coded by each BV in the BV candidate list after the iterative operation;

[0244] The BV corresponding to the minimum value in the second coding overhead is used as the BV of the image block to be encoded.

[0245] Specifically, for AVS3, when determining to use Intra Affine Mode for encoding, in order to determine the optimal BV, here, the corresponding encoding overhead can be calculated for each BV, and then the BV corresponding to the minimum value is selected as the BV of the image block to be encoded, so as to reduce the encoding overhead when using Intra Affine Mode for encoding.

[0246] Fig.22A A schematic diagram of an optional process for determining a coding mode provided in an embodiment of the present application is as follows: Fig.22A As shown, the specific implementation steps are as follows:

[0247] S22A1: input the current coding unit; execute S22A2 and S22A3;

[0248] S22A2: Uses conventional Intra Mode for encoding and executes S22A6;

[0249] S22A3: Determine the distance of 16 pixels at which the length H and the width W of the image block to be encoded are both greater than or equal to;

[0250] S22A4: If both are greater than or equal to, use Intra Affine Mode for encoding. Specifically, first establish an initial BV candidate list;

[0251] S22A5: iteratively search for the best BV starting from the initial BV candidate list;

[0252] S22A6: Calculate the coding overheads of Intra Mode and Intra Affine Mode respectively, and finally select the coding mode corresponding to the minimum value of the coding overhead from the coding overheads, and determine it as the final coding mode.

[0253] That is to say, for AVS3, after obtaining the image block to be encoded, the conventional Intra Mode is used for encoding. At the same time, when the length and width of the image block to be encoded are both greater than or equal to the distance of 16 pixels, Intra AffineMode is used for encoding (through iterative operations, the BV corresponding to the minimum coding overhead can be selected for encoding), and then the coding overheads of Intra Mode and Intra Affine Mode are calculated respectively, and finally the coding mode corresponding to the minimum coding overhead is selected from the coding overhead and determined as the final coding mode.

[0254] Fig. 22B A schematic diagram of another optional process for determining a coding mode provided in an embodiment of the present application, such as Fig. 22B As shown, the specific implementation steps are as follows:

[0255] S22B1: input the current coding unit; execute S22B2, S22B3 and S22B7;

[0256] S22B2: Use the conventional Intra Mode for encoding and execute S22B8;

[0257] S22B3: Determine the distance of 16 pixels at which the length H and the width W of the image block to be encoded are both greater than or equal to;

[0258] S22B4: If both are greater than or equal to, use Intra Affine Mode for encoding. Specifically, first establish an initial BV candidate list;

[0259] S22B5: iteratively search for the best BV starting from the initial BV candidate list, and execute S22B8;

[0260] S22B6: Determine the distance of 16 pixels at which the length H and the width W of the image block to be encoded are both less than or equal to;

[0261] S22B7: Use the conventional IBC Mode for encoding and execute S22B8;

[0262] S22B8: Calculate the coding overheads of Intra Mode, IBC Mode, and Intra Affine CPR Mode respectively, and finally select a coding mode corresponding to a minimum value of the coding overhead from the coding overheads, and determine it as the final coding mode.

[0263] That is to say, for VVC, after obtaining the image block to be encoded, the conventional Intra Mode is used for encoding, and the IBC Mode is used for encoding when the length and width of the image block to be encoded are both less than or equal to the distance of 64 pixels. At the same time, the Intra Affine CPR Mode is used for encoding when the length and width of the image block to be encoded are both greater than or equal to the distance of 16 pixels (through iterative calculation, the BV corresponding to the minimum coding overhead can be selected for encoding), and then the coding overheads of Intra Mode, IBC Mode and Intra Affine CPR Mode are calculated respectively, and finally the coding mode corresponding to the minimum coding overhead is selected from all the coding overheads, and determined as the final coding mode.

[0264] In an optional embodiment, before S1004, the method may further include:

[0265] Determine whether the control point of the reference image block pointed to by each BV in the BV candidate list after the iterative operation is within the encoded image block;

[0266] From the BV candidate list after the iterative operation, delete the BV whose control point of the reference image block pointed to in the BV candidate list after the iterative operation is not in the encoded image block.

[0267] Here, whether the BV in the BV candidate list is legal is determined by judging whether the control point of the reference image block pointed to by each BV in the BV candidate list after the iterative operation is within the encoded image block. Only the BV whose control point of the reference image block pointed to by the BV candidate list after the iterative operation is within the encoded image block is a legal BV, otherwise it is illegal.

[0268] For example, in the Intra Affine Mode, the current frame itself needs to be used as the only reference frame. In order to ensure successful decoding, the reference unit must be in the encoded area of ​​the current frame.

[0269] Therefore, in Intra Affine Mode, it is necessary to perform a legitimacy check on the searched BV, including the initial BV in the BV candidate list mentioned above, as well as the BV and optimal BV that are continuously updated during the iterative search process; if the legitimacy check fails, it means that the reference pixel point may exceed the image boundary or be located in the uncoded area, and the available reference pixel value cannot be obtained at the decoding end, resulting in decoding confusion.

[0270] Taking the four-parameter Affine as an example, assuming that the width and length of the image are W_pic and H_pic respectively, and the BVs of the control points LT (upper left corner) and RT (upper right corner) of the current coding unit are (v 0x ,v 0y ) and (v 1x ,v 1y ), the width and length of the current coding unit are w and h, the position of any pixel in the coding unit is (x, y), and the position of the upper left corner pixel is (X PU ,Y PU ), in order to limit the pixels in the reference unit to be in the encoded area and not beyond the image boundary, the legality restriction conditions are as follows:

[0271]

[0272]

[0273] For each BV obtained in the Intra Affine Mode, only those that satisfy the above formula are considered legal and usable, otherwise they are discarded.

[0274] The validity check of the six parameters is basically the same and will not be repeated here.

[0275] It should be noted that the legitimacy check of BV in Intra Affine CPR Mode is similar to that in Intra Affine Mode, and will not be described in detail here.

[0276] S1005: According to the BV of the image block to be encoded, a preset affine motion model is called to calculate a reference image block of the image block to be encoded, and a pixel value of the reference image block is used as a prediction value of the image block to be encoded.

[0277] In this way, after determining the BV of the image block to be encoded, the BV of each sub-block in the image block to be encoded can be calculated using formula (1) and formula (2), and then for each sub-block, the reference image block of each sub-block can be determined according to the BV of each sub-block, and the pixel value of the reference image block of each sub-block is used as the prediction value of each sub-block, so that the prediction value of the image block to be encoded can be determined.

[0278] In the embodiment of the present application, the coding mode of Intra Affine Mode and the coding mode of Intra Affine CPR Mode are introduced. In order to better implement the coding mode of Intra Affine Mode and the coding mode of Intra Affine CPR Mode in the encoder and the decoder, the following method is adopted:

[0279] When the Intra Affine Mode is executed at the encoder, in order to mark or infer the mode, other encoding units can refer to the encoding units with the Intra Affine Mode, or the decoding end can decode the mode.

[0280] In the embodiment of the present application, there is no need to transmit a Flag in the bitstream to mark the proposed Intra AffineMode. As described above, when the encoding end enters the Intra Affine Mode, the current image type PictureType is set to 1, that is, it becomes a P frame (PictureType=0 represents an I frame, 1 represents a P frame, and 2 represents a B frame). At this time, the prediction mode of the encoding unit is set to PRED_No_Constraint (intra-frame prediction and inter-frame prediction can be used), and the current image is set to the only reference frame for prediction (the current frame is inserted into the reference picture list 0), and the AffineFlag of the encoding unit is set to true.

[0281] Therefore, whether at the encoding end or the decoding end, as long as it is detected that the prediction mode of a certain coding unit is PRED_No_Constraint, "the reference frame is the current frame", and "AffineFlag is true", it can be inferred that the coding unit adopts the Intra Affine Mode proposed in this application; Fig.23 A flowchart of a method for inferring an optional coding mode provided in an embodiment of the present application is shown as follows: Fig.23 As shown, the specific inference process is as follows:

[0282] S2301: Determine whether the prediction mode is PRED_No_Constraint, if yes, execute S2302, otherwise execute S2303;

[0283] S2302: Determine whether the reference frame is the current frame, if yes, execute S2304, otherwise, select Inter Mode;

[0284] S2303: Determine whether the prediction mode is PRED_Intra_Only, if yes, select Intra Mode.

[0285] S2304: Determine whether AffineFlag is equal to true, and if so, select Intra Affine Mode.

[0286] In this embodiment, there is no need to transmit additional Flags, and only the original flags in the multiplexed code stream are used, which is beneficial to reducing the code rate and improving the coding efficiency.

[0287] Similarly, for the Intra Affine CPR Mode in VVC, in order to mark or infer the mode, other PUs can refer to the PU with the Intra Affine CPR Mode, or the decoding end can decode it.

[0288] In the embodiment of the present application, there is no need to transmit a Flag in the bitstream to mark the proposed Intra Affine CPR Mode. Instead, when the encoding end enters the Intra Affine CPR Mode, the current image type is changed from I frame to P frame, and the Flag_predMode of the encoding unit is set to MODE_INTER (inter frame), and the current image is set as the only reference frame for prediction; in addition, the Flag_Affine of the encoding unit is set to true. Therefore, whether at the encoding end or the decoding end, as long as it is detected that "Flag_predMode of a coding unit is MODE_INTER", "the reference frame is the current frame" and "Flag_Affine is true", it can be inferred that the coding unit adopts the Intra Affine CPR Mode proposed in this article.

[0289] Fig.24 A flowchart of another optional method for inferring a coding mode provided in an embodiment of the present application is shown as follows: Fig.24 As shown, the specific inference process is as follows:

[0290] S2401: Determine whether Flag_predMode is MODE_INTER, if yes, execute S2402, if no, execute and select Intra Mode;

[0291] S2402: Determine whether the reference frame is the current frame, if yes, execute S2403, otherwise select Inter Mode;

[0292] S2403: Determine whether Flag_Affine is true, if so, select Intra Affine CPR Mode, otherwise select IBC mode.

[0293] In this way, there is no need to transmit additional Flags, and only the original flags in the multiplexed code stream can be used, which is conducive to reducing the bit rate and improving the coding efficiency.

[0294] The embodiment of the present application provides a method for determining a prediction value, including: an encoder obtains an image block to be encoded and an encoded image block from a current image frame, constructs a BV candidate list for the image block to be encoded according to the encoded image block, performs an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation, selects the BV of the image block to be encoded from the BV candidate list after the iterative operation, calls a preset affine motion model according to the BV of the image block to be encoded, calculates a reference image block of the image block to be encoded, and uses the pixel value of the reference image block as the prediction value of the image block to be encoded; that is, in the embodiment of the present application, by constructing a BV candidate list for the image block to be encoded and performing an iterative operation on each BV in the BV candidate list, the BV of the image block to be encoded can be selected from the BV candidate list after the iterative operation, so that a more accurate BV can be determined for the image block to be encoded, and then the affine motion model is called so that the prediction value is predicted by using affine motion compensation in intra-frame prediction encoding, so that the determined prediction value is closer to the true value, effectively improving the encoding efficiency.

[0295] Embodiment 2

[0296] Based on the same invention concept, Fig.25 A schematic diagram of the structure of an optional encoder proposed in an embodiment of the present application is shown in FIG. Fig.25 As shown, the encoder proposed in the embodiment of the present application may include an acquisition module 251, a construction module 252, an iteration module 253, a selection module 254 and a determination module 255.

[0297] The acquisition module 251 is used to acquire the image block to be encoded and the encoded image block from the current image frame;

[0298] A construction module 252, configured to construct a block vector BV candidate list for the image block to be encoded according to the encoded image block;

[0299] Iteration module 253, used for performing iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after iterative operation;

[0300] A selection module 254, configured to select a BV of the image block to be encoded from the BV candidate list after the iterative operation;

[0301] The determination module 255 is used to call a preset affine motion model according to the BV of the image block to be encoded, calculate a reference image block of the image block to be encoded, and use the pixel value of the reference image block as the prediction value of the image block to be encoded.

[0302] Further, when the encoding mode adopted by the encoder is the intra-frame mode, the construction module 252 is specifically used to:

[0303] Acquire a preset area adjacent to the image block to be encoded;

[0304] According to the starting point of the control point set for the preset area and the preset search step length, the preset area is searched to find the control point for the image block to be encoded;

[0305] Calculate the searched BV for the image block to be encoded based on the searched control points and the control points of the image block to be encoded;

[0306] The searched BV is added to the BV candidate list.

[0307] Further, the construction module 252 is specifically used for:

[0308] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0309] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0310] The BV candidate list of adjacent image blocks is constructed from the searched BVs.

[0311] Furthermore, the construction module 252 is also specifically used for:

[0312] After selecting the adjacent image blocks of the image block to be encoded from the encoded image blocks, adding the BV candidate list of the adjacent image blocks to the BV candidate list;

[0313] The BV candidate list of adjacent image blocks is constructed from the searched BV and / or the BV candidate list of adjacent image blocks of the adjacent image blocks.

[0314] Further, when the encoding mode adopted by the encoder is the intra-frame mode and the intra-frame block copy IBC mode, the construction module 252 is specifically used to:

[0315] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0316] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0317] The BV candidate list of the adjacent image blocks is constructed by the BV of the reference image blocks of the adjacent image blocks.

[0318] Further, the construction module 252 is specifically used for:

[0319] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0320] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0321] The BV candidate list of adjacent image blocks is constructed from the BV candidate lists of adjacent image blocks of adjacent image blocks.

[0322] Further, the construction module 252 is specifically used for:

[0323] Acquire a preset area adjacent to the image block to be encoded;

[0324] According to the starting point of the control point set for the preset area and the preset search step length, the preset area is searched to find the control point for the image block to be encoded;

[0325] Calculate the searched BV for the image block to be encoded based on the searched control points and the control points of the image block to be encoded;

[0326] The searched BV is added to the BV candidate list.

[0327] Further, the construction module 252 is specifically used for:

[0328] Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded;

[0329] Add the BV candidate list of the adjacent image block to the BV candidate list;

[0330] The BV candidate list of the adjacent image blocks is constructed by the BV of the reference image block of the adjacent image block, and / or the BV candidate list of the adjacent image block of the adjacent image block, and / or the searched BV.

[0331] Further, the construction module 252 is specifically used for:

[0332] According to the BV of the adjacent image block, the control point of the reference image block of the adjacent image block is calculated;

[0333] The BV of the image block to be encoded and the reference image block of the adjacent image block is calculated according to the control point of the reference image block of the adjacent image block and the control point of the image block to be encoded;

[0334] The BVs of the image block to be encoded and the reference image blocks of the adjacent image blocks are added to the BV candidate list.

[0335] Further, the construction module 252 is specifically used for:

[0336] Based on the control point of the image block to be encoded, the BV of the adjacent image block is used to calculate an indirect reference image block of the image block to be encoded;

[0337] According to the BV of the adjacent image block, the control point of the indirect reference image block, the BV of the indirect reference image block and the control point of the image block to be encoded, a vector addition and subtraction algorithm is used to calculate the BV of the image block to be encoded and the indirect reference image block;

[0338] The BVs of the image block to be encoded and the indirect reference image block are added to the BV candidate list.

[0339] Further, the construction module 252 is specifically used for:

[0340] Determine whether the length and width of the image block to be encoded are both greater than or equal to a distance of 16 pixel values;

[0341] When the length and width of the image block to be encoded are both greater than or equal to the distance of 16 pixels, a block vector BV candidate list is constructed for the image block to be encoded according to the encoded image block;

[0342] When the length or width of the image block to be encoded is less than the distance of 16 pixels, the image block to be encoded is encoded in intra-frame mode to obtain a predicted value of the image block to be encoded.

[0343] Further, the construction module 252 is specifically used for:

[0344] After calculating the searched BV for the image block to be encoded according to the searched control point and the control point of the image block to be encoded, calculating the first encoding cost generated by respectively encoding the image block to be encoded using each BV in the searched BV;

[0345] Selecting a preset number of BVs from the first coding overhead in descending order;

[0346] A preset number of BVs are added to the BV candidate list.

[0347] Furthermore, the iteration module 253 is specifically used for:

[0348] Determine whether the number of iterations i is less than or equal to a preset iteration threshold; wherein the initial value of the number of iterations i is 1;

[0349] When i is less than or equal to the iteration threshold, each BV in the BV candidate list is substituted into the error formula between the real value of the image block to be encoded and the predicted value of the image block to be encoded, and the least squares algorithm is used to update the parameters of the affine motion model in the error formula and the BV candidate list;

[0350] When the number of iterations is greater than a preset iteration threshold, an updated BV candidate list is obtained as the BV candidate list after the iterative operation, and the parameters of the affine motion model in the updated error formula are used to represent the parameters in the affine motion model;

[0351] i is updated to i+1, and the process returns to execute to determine whether the number of iterations i is less than or equal to the preset number of iterations.

[0352] Further, the selection module 253 is specifically used for:

[0353] Calculate the second coding overhead generated by respectively coding the image block to be coded by each BV in the BV candidate list after the iterative operation;

[0354] The BV corresponding to the minimum value in the second coding overhead is used as the BV of the image block to be encoded.

[0355] Furthermore, the encoder is also used for:

[0356] Before selecting the BV of the image block to be encoded from the BV candidate list after the iterative operation, determining whether the control point of the reference image block pointed to by each BV in the BV candidate list after the iterative operation is within the encoded image block;

[0357] From the BV candidate list after the iterative operation, delete the BV whose control point of the reference image block pointed to in the BV candidate list after the iterative operation is not in the encoded image block.

[0358] Fig.26 A schematic diagram of the structure of another optional encoder proposed in the embodiment of the present application, such as Fig.26 As shown, the encoder 2600 proposed in the embodiment of the present application may also include a processor 261 and a storage medium 262 storing executable instructions of the processor 261. The storage medium 262 relies on the processor 261 to perform operations through a communication bus 263. When the instructions are executed by the processor 261, the method for determining the prediction value described in one or more of the above embodiments is executed.

[0359] It should be noted that in actual application, the various components in the terminal are coupled together through the communication bus 263. It is understandable that the communication bus 263 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 263 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.26 Various buses are labeled as communication buses 263.

[0360] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the method for determining the prediction value described in one or more of the above embodiments.

[0361] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0362] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0363] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0364] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0365] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0366] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0367] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0368] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0369] Industrial Applicability

[0370] The embodiment of the present application provides a method for determining a prediction value, an encoder, and a computer storage medium. The encoder obtains an image block to be encoded and an encoded image block from a current image frame, constructs a BV candidate list for the image block to be encoded based on the encoded image block, performs an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm, obtains a BV candidate list after the iterative operation, selects the BV of the image block to be encoded from the BV candidate list after the iterative operation, calls a preset affine motion model based on the BV of the image block to be encoded, calculates a reference image block of the image block to be encoded, and uses the pixel value of the reference image block as the prediction value of the image block to be encoded; in this way, the determined prediction value is closer to the true value, which effectively improves the encoding efficiency.

Claims

1. A method for determining a predicted value, wherein: The method is applied in an encoder, and the method comprises: Obtaining the image block to be encoded and the encoded image block from the current image frame; According to the encoded image block, construct a block vector BV candidate list for the image block to be encoded; Performing an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation; wherein the preset iterative algorithm includes: substituting each BV in the BV candidate list into an error formula between a true value of the image block to be encoded and a predicted value of the image block to be encoded, and updating the parameters of the affine motion model in the error formula and the BV candidate list; Selecting the BV of the image block to be encoded from the BV candidate list after the iterative operation; According to the BV of the image block to be encoded, a preset affine motion model is called to calculate a reference image block of the image block to be encoded, and a pixel value of the reference image block is used as a prediction value of the image block to be encoded.

2. The method according to claim 1, wherein: When the encoding mode adopted by the encoder is the intra-frame mode, constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block includes: Acquire a preset area adjacent to the image block to be encoded; According to the starting point of the control point set for the preset area and the preset search step length, the preset area is searched to find the control point for the image block to be encoded; Calculating a searched BV for the image block to be encoded according to the searched control point and the control point of the image block to be encoded; The searched BV is added to the BV candidate list.

3. The method according to claim 2, wherein: The step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block comprises: Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded; Adding the BV candidate list of the adjacent image block to the BV candidate list; The BV candidate list of the adjacent image blocks is constructed by the searched BVs.

4. The method according to claim 3, wherein: After selecting the adjacent image blocks of the image block to be encoded from the encoded image blocks, the method further includes: Adding the BV candidate list of the adjacent image block to the BV candidate list; The BV candidate list of the adjacent image blocks is constructed by the searched BV and / or the BV candidate list of the adjacent image blocks of the adjacent image blocks.

5. The method according to claim 1, wherein: When the encoding mode adopted by the encoder is the intra-frame mode and the intra-frame block copy IBC mode, the step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block includes: Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded; Adding the BV candidate list of the adjacent image block to the BV candidate list; The BV candidate list of the adjacent image blocks is constructed by the BV of the reference image blocks of the adjacent image blocks.

6. The method according to claim 5, wherein: The step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block comprises: Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded; Adding the BV candidate list of the adjacent image block to the BV candidate list; The BV candidate list of the adjacent image blocks is constructed from the BV candidate lists of the adjacent image blocks of the adjacent image blocks.

7. The method according to claim 5, wherein: The step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block comprises: Acquire a preset area adjacent to the image block to be encoded; According to the starting point of the control point set for the preset area and the preset search step length, the preset area is searched to find the control point for the image block to be encoded; Calculating a searched BV for the image block to be encoded according to the searched control point and the control point of the image block to be encoded; The searched BV is added to the BV candidate list.

8. The method according to any one of claims 5 to 7, wherein: The step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block comprises: Selecting, from the encoded image blocks, image blocks adjacent to the image block to be encoded; Adding the BV candidate list of the adjacent image block to the BV candidate list; The BV candidate list of the adjacent image blocks is constructed by the BV of the reference image block of the adjacent image block, and / or the BV candidate list of the adjacent image blocks of the adjacent image block, and / or the searched BV.

9. The method according to claim 8, wherein: The step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block comprises: Calculating and obtaining a control point of a reference image block of the adjacent image block according to the BV of the adjacent image block; Calculate the BV of the image block to be encoded and the reference image block of the adjacent image block according to the control point of the reference image block of the adjacent image block and the control point of the image block to be encoded; The BV of the image block to be encoded and the reference image block of the adjacent image block is added to the BV candidate list.

10. The method according to claim 8, wherein: The step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block comprises: Based on the control point of the image block to be encoded, using the BV of the adjacent image block, calculate an indirect reference image block of the image block to be encoded; Calculating the BVs of the image block to be encoded and the indirect reference image block using a vector addition and subtraction algorithm according to the BV of the adjacent image block, the control point of the indirect reference image block, the BV of the indirect reference image block, and the control point of the image block to be encoded; Add the BV of the to-be-encoded image block and the indirect reference image block to the BV candidate list.

11. The method according to claim 1, wherein: The step of constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block comprises: Determine whether the length and width of the image block to be encoded are both greater than or equal to a distance of 16 pixels; When the length and width of the image block to be encoded are both greater than or equal to the distance of 16 pixels, constructing a block vector BV candidate list for the image block to be encoded according to the encoded image block; When the length or width of the image block to be encoded is smaller than a distance of 16 pixels, the image block to be encoded is encoded in intra-frame mode to obtain a predicted value of the image block to be encoded.

12. The method according to claim 2 or 7, wherein: After calculating the searched BV for the image block to be encoded according to the searched control point and the control point of the image block to be encoded, the method further includes: Calculate a first encoding cost generated by respectively encoding the image block to be encoded using each BV in the searched BVs; Selecting a preset number of BVs from the first coding overhead in descending order; A preset number of BVs are added to the BV candidate list.

13. The method according to claim 1, wherein: Performing an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation, including: Determine whether the number of iterations i is less than or equal to a preset iteration threshold; wherein the initial value of the number of iterations i is 1; When i is less than or equal to the iteration threshold, a least squares algorithm is used to update the parameters of the affine motion model in the error formula and the BV candidate list; When the number of iterations is greater than the iteration threshold, an updated BV candidate list is obtained as the BV candidate list after the iterative operation, and the parameters of the affine motion model in the updated error formula are used to update the parameters in the affine motion model; i is updated to i+1, and the process returns to execute the determination of whether the number of iterations i is less than or equal to a preset iteration threshold.

14. The method according to claim 1, wherein: The selecting the BV of the image block to be encoded from the BV candidate list after the iterative operation comprises: Calculate the second encoding cost generated by respectively encoding the image block to be encoded using each BV in the BV candidate list after the iterative operation; The BV corresponding to the minimum value of the second coding overhead is used as the BV of the image block to be encoded.

15. The method according to claim 1, wherein: Before selecting the BV of the image block to be encoded from the BV candidate list after the iterative operation, the method further includes: Determine whether a control point of a reference image block pointed to by each BV in the BV candidate list after the iterative operation is within the encoded image block; From the BV candidate list after the iterative operation, delete the BV whose control point of the reference image block pointed to in the BV candidate list after the iterative operation is not in the encoded image block.

16. An encoder, wherein: The encoder comprises: An acquisition module, used for acquiring the image blocks to be encoded and the encoded image blocks from the current image frame; A construction module, configured to construct a BV candidate list for the image block to be encoded according to the encoded image block; An iteration module is used to perform an iterative operation on each BV in the BV candidate list according to a preset iterative algorithm to obtain a BV candidate list after the iterative operation; the preset iterative algorithm includes: substituting each BV in the BV candidate list into an error formula between a true value of the image block to be encoded and a predicted value of the image block to be encoded, and updating the parameters of the affine motion model in the error formula and the BV candidate list; A selection module, configured to select the BV of the image block to be encoded from the BV candidate list after the iterative operation; The determination module is used to call a preset affine motion model according to the BV of the image block to be encoded, calculate a reference image block of the image block to be encoded, and use the pixel value of the reference image block as the prediction value of the image block to be encoded.

17. An encoder, wherein: The encoder comprises: A processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations via a communication bus, and when the instructions are executed by the processor, the method for determining the prediction value described in any one of claims 1 to 15 is executed.

18. A computer-readable storage medium, wherein: Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the method for determining the prediction value described in any one of claims 1 to 15 above.

Images