Inter-frame prediction method, encoder, decoder and computer readable storage medium

Abstract

The embodiment of the invention discloses an inter-frame prediction method, an encoder, a decoder and a computer readable storage medium, and the method comprises the steps: analyzing a code stream, and determining a prediction decoding parameter of a current block; determining a GPM mode parameter and curve division information of the current block when the prediction decoding parameter indicates that an inter-frame prediction of the current block is determined using an inter-frame geometric division prediction mode GPM; determining a first reference block and a second reference block based on the GPM mode parameter; determining a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve division information; and determining a prediction value of the current block based on the first prediction parameter and the second prediction parameter.

Description

[0001] This application requires a divisional application filed with the China Patent Office on March 31, 2020, with application number 202010246244.3 and invention name “Inter-frame prediction method, encoder, decoder and computer-readable storage medium”. Technical Field

[0002] The embodiments of the present application relate to video coding technology, and relate to but are not limited to an inter-frame prediction method, an encoder, a decoder, and a computer-readable storage medium. Background Art

[0003] In the process of video encoding and decoding, in addition to intra-frame prediction, inter-frame prediction can also be used for encoding and decoding the current block. Inter-frame prediction can include motion estimation and motion compensation. For motion compensation, the inter-frame geometric partitioning prediction mode (GPM) can be used to divide the current block between frames into two non-rectangular partitions (or two blocks) for prediction and weighted fusion, thereby obtaining the prediction value of the current block.

[0004] At present, in the prediction process of GPM, a rectangular block is divided into two blocks by a straight line. The geometric division here may be the division of coding units or the division of prediction units, which are collectively referred to as blocks. In this way, the divided blocks are triangular or trapezoidal. The straight line that divides the rectangular block into two blocks can have different angles and different positions, so that a variety of combinations of triangles and trapezoids can be divided. Such geometric division can make the edge of the block closer to the edge of the object, or close to the edge of two partitions with different motion vectors, thereby improving the encoding performance.

[0005] However, such (straight line) geometric division cannot completely fit the edge of an object, or the edge of two partitions with different motion vectors. Transition will be used for the edge area divided by the geometric division. For some scenes, such as the presence of a large number of curves and arcs in natural objects, such straight line division is still too rigid, thus affecting the performance of encoding and decoding. Summary of the invention

[0006] The embodiments of the present application provide an inter-frame prediction method, an encoder, a decoder, and a computer-readable storage medium, which can improve the accuracy of encoding and decoding.

[0007] The technical solution of this application is implemented as follows:

[0008] The embodiment of the present application provides an inter-frame prediction method, which is applied in a decoder and includes:

[0009] Parse the bitstream and determine the prediction decoding parameters of the current block;

[0010] When the prediction decoding parameter indicates that the inter-frame prediction of the current block is determined using the inter-frame geometric partitioning prediction mode GPM, determining the GPM mode parameters and curve partitioning information of the current block;

[0011] Based on the GPM mode parameters, determining a first reference block and a second reference block;

[0012] Determine, based on the GPM mode parameter and the curve division information, a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block;

[0013] Based on the first prediction parameter and the second prediction parameter, a prediction value of the current block is determined.

[0014] The embodiment of the present application further provides an inter-frame prediction method, which is applied in an encoder and includes:

[0015] Determining prediction mode parameters for the current block;

[0016] When the prediction mode parameter indicates that the inter-frame prediction of the current block is determined using the inter-frame geometric partitioning prediction mode GPM, determining the GPM mode parameter and curve partitioning information of the current block;

[0017] Based on the GPM mode parameters, determining a first reference block and a second reference block;

[0018] Determine, based on the GPM mode parameter and the curve division information, a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block;

[0019] Based on the first prediction parameter and the second prediction parameter, a prediction value of the current block is determined.

[0020] The present application provides a decoder, including:

[0021] A decoding unit, used for parsing the bitstream and determining the prediction decoding parameters of the current block;

[0022] A first determining unit is configured to, when the prediction decoding parameter indicates that the inter-frame geometric partitioning prediction mode GPM is used to determine the inter-frame prediction of the current block, determine the GPM mode parameter and curve partitioning information of the current block; determine a first reference block and a second reference block based on the GPM mode parameter; determine a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve partitioning information;

[0023] The first prediction unit is used to determine the prediction value of the current block based on the first prediction parameter and the second prediction parameter.

[0024] The present application provides an encoder, including:

[0025] a second determination unit, configured to determine a prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter-frame prediction of the current block is determined using an inter-frame geometric partitioning prediction mode GPM, determine the GPM mode parameter and curve partitioning information of the current block; determine a first reference block and a second reference block based on the GPM mode parameter; determine a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve partitioning information;

[0026] The second prediction unit is used to determine the prediction value of the current block based on the first prediction parameter and the second prediction parameter.

[0027] An embodiment of the present application also provides a decoder, comprising: a first processor and a first memory storing executable instructions of the first processor, wherein the first memory relies on the first processor to perform operations through a communication bus, and when the executable instructions are executed by the first processor, the inter-frame prediction method described in any one of claims 1 to 18 is executed.

[0028] An embodiment of the present application also provides an encoder, comprising: a second processor and a second memory storing executable instructions of the second processor, the second memory relying on the second processor to perform operations through a communication bus, and when the executable instructions are executed by the second processor, the inter-frame prediction method described in any one of claims 19 to 42 is executed.

[0029] An embodiment of the present application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by one or more first processors, the first processors execute the inter-frame prediction method on the decoder side; or, when the executable instructions are executed by one or more second processors, the second processors execute the inter-frame prediction method on the encoder side.

[0030] The embodiment of the present application provides an inter-frame prediction method, an encoder, a decoder and a computer-readable storage medium, including: parsing a bitstream to determine prediction decoding parameters of a current block; when the prediction decoding parameters indicate that the inter-frame prediction of the current block is determined using an inter-frame geometric partitioning prediction mode GPM, determining the GPM mode parameters and curve partitioning information of the current block; determining a first reference block and a second reference block based on the GPM mode parameters; determining a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block based on the GPM mode parameters and the curve partitioning information; determining a prediction value of the current block based on the first prediction parameter and the second prediction parameter. That is, in the embodiment of the present application, by adopting the curve partitioning method in the GPM mode to predict the current block, the divided boundary is more closely aligned with the edge of the object or the edge of two parts with different motion vectors, thereby improving the accuracy of encoding and decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic block diagram of a video encoding system provided in an embodiment of the present application;

[0032] Figure 2 A schematic block diagram of a video decoding system provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of an implementation flow of an inter-frame prediction method provided in an embodiment of the present application;

[0034] Figure 4a-4i A schematic diagram of an exemplary geometric partitioning method used by VVC provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of division angles of an exemplary VVC geometric division provided in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of an exemplary transformation from a straight line dividing line to a curved line dividing line provided in an embodiment of the present application;

[0037] Figure 7 Schematic diagram of another implementation process of the inter-frame prediction method provided in the embodiment of the present application Figure 1 ;

[0038] Figure 8 Schematic diagram of another implementation process of the inter-frame prediction method provided in the embodiment of the present application Figure 2 ;

[0039] Fig. 9 A schematic diagram of a decoder provided in an embodiment of the present application Figure 1 ;

[0040] Fig.10A schematic diagram of a decoder provided in an embodiment of the present application Figure 2 ;

[0041] Fig.11 A schematic diagram of the structure of an encoder provided in an embodiment of the present application Figure 1 ;

[0042] Fig.12 A schematic diagram of the structure of an encoder provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0044] The present application provides a video encoding system, such as Figure 1 As shown, the video encoding system 11 includes:

[0045] Transformation unit 111, quantization unit 112, mode selection and coding control logic unit 113, intra-frame prediction unit 114, inter-frame prediction unit 115 (including: motion compensation and motion estimation), inverse quantization unit 116, inverse transformation unit 117, loop filtering unit 118, coding unit 119 and decoded image buffer unit 110; for the input original video signal, through the coding tree block (Coding Tree A video reconstruction block can be obtained by dividing the video reconstruction block into a video unit (CTU), and the coding mode is determined by the mode selection and coding control logic unit 113. Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transformation unit 111 and the quantization unit 112, including transforming the residual information from the pixel domain to the transform domain, and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame prediction unit 114 is used to perform intra-frame prediction on the video reconstruction block; wherein the intra-frame prediction unit 114 is used to determine the optimal intra-frame prediction mode (i.e., the target prediction mode) of the video reconstruction block; the inter-frame prediction unit 115 is used to perform inter-frame prediction coding of the received video reconstruction block relative to one or more blocks in one or more reference frames to provide time prediction information; wherein, motion estimation is used to generate motion estimation information. The motion vector process can estimate the motion of the video reconstruction block, and then the motion compensation is performed based on the motion vector determined by the motion estimation; after determining the inter-frame prediction mode, the inter-frame prediction unit 115 is also used to provide the selected inter-frame prediction data to the encoding unit 119, and the calculated motion vector data is also sent to the encoding unit 119; in addition, the inverse quantization unit 116 and the inverse transformation unit 117 are used to reconstruct the video reconstruction block, reconstruct the residual block in the pixel domain, and the reconstructed residual block is removed by the loop filter unit 118. Then, the reconstructed residual block is added to a predictive block in the frame of the decoded image cache unit 110 to generate a reconstructed video reconstruction block; the encoding unit 119 is used to encode various encoding parameters and quantized transform coefficients. The decoded image cache unit 110 is used to store the reconstructed video reconstruction block for prediction reference. As the video image encoding proceeds, new reconstructed video reconstruction blocks will be continuously generated, and these reconstructed video reconstruction blocks will be stored in the decoded image cache unit 110.

[0046] The embodiment of the present application provides a video decoding system. Figure 2 FIG. 1 is a schematic diagram of the structure of the video encoding system according to an embodiment of the present application. Figure 2 As shown, the video encoding system 12 includes:

[0047] Decoding unit 121, inverse transform unit 127, inverse quantization unit 122, intra-frame prediction unit 123, motion compensation unit 124, loop filter unit 125 and decoded image cache unit 126; after the input video signal is encoded by the video encoding system 11, the code stream of the video signal is output; the code stream is input into the video decoding system 12, firstly passes through the decoding unit 121, and is used to obtain the decoded transform coefficient; the transform coefficient is processed by the inverse transform unit 127 and the inverse quantization unit 122 to generate a residual block in the pixel domain; the intra-frame prediction unit 123 can be used to generate prediction data of the current video decoding block based on the determined intra-frame prediction direction and the data of the previously decoded block from the current frame or picture; the motion compensation unit 124 determines prediction information for a video decoding block by analyzing motion vectors and other associated syntax elements, and uses the prediction information to generate a predictive block of the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform unit 127 and the inverse quantization unit 122 with the corresponding predictive block generated by the intra-frame prediction unit 123 or the motion compensation unit 124; the decoded video signal passes through a loop filter unit 125 to remove block effect artifacts, thereby improving video quality; the decoded video block is then stored in a decoded image cache unit 126, which stores reference images for subsequent intra-frame prediction or motion compensation, and is also used for output of video signals to obtain the restored original video signal.

[0048] An inter-frame prediction method provided in an embodiment of the present application mainly acts on the inter-frame prediction unit 215 of the video encoding system 11 and the inter-frame prediction unit of the video decoding system 12, namely, the motion compensation unit 124; that is, if the video encoding system 11 can obtain a better prediction effect through the inter-frame prediction method provided in an embodiment of the present application, then, correspondingly, at the decoding end, the video decoding recovery quality can also be improved.

[0049] Based on this, the technical solution of the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. Before the detailed description, it should be noted that the "first", "second", "third", etc. mentioned throughout the specification are only for distinguishing different features and do not have the function of limiting priority, sequence, size relationship, etc.

[0050] The embodiment of the present application provides an inter-frame prediction method, which is applied to a video decoding device, i.e., a decoder. The function implemented by the method can be implemented by calling a program code by a first processor in the video decoding device, and of course the program code can be stored in a first memory. It can be seen that the video decoding device at least includes a first processor and a first memory.

[0051] Figure 3FIG. 1 is a schematic diagram of an implementation flow of an inter-frame prediction method according to an embodiment of the present application. Figure 3 As shown, the method includes:

[0052] S101, parsing a bit stream to determine prediction decoding parameters of a current block.

[0053] S102: When the prediction decoding parameter indicates that the inter-frame prediction of the current block is determined by using the GPM, determine the GPM mode parameter and curve division information of the current block.

[0054] S103. Determine a first reference block and a second reference block based on GPM mode parameters.

[0055] S104. Determine a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block based on the GPM mode parameters and the curve division information.

[0056] S105. Determine a prediction value of the current block based on the first prediction parameter and the second prediction parameter.

[0057] In the embodiment of the present application, during the decoding process, the decoder can parse out information such as the encoding method used by the encoder when encoding, that is, the prediction decoding parameters for the current block from the bit stream. Based on the diversity of current encoding methods, the prediction modes decoded by the decoder may be diverse. The inter-frame prediction method provided in the embodiment of the present application is a scenario that can only be implemented when the encoding adopts the GPM prediction mode.

[0058] When the decoder parses the bitstream to obtain the prediction decoding parameters indicating the use of the inter-frame geometric partitioning prediction mode GPM to determine the inter-frame prediction of the current block, the inter-frame prediction method provided in the embodiment of the present application is adopted to parse the GPM mode parameters and curve partitioning information of the current block used in encoding from the bitstream.

[0059] The GPM mode parameters include the GPM division mode and motion related information. The GPM division mode can be 64.

[0060] In the embodiment of the present application, the decoder decodes the prediction mode of the current block. If the current block uses GPM, GPM needs to decode the partition mode of GPM. If GPM uses a mask method, the information of the MV used by the two partitioned parts of the current block or other information related to GPM is decoded.

[0061] For the curve division mode, the decoder can directly determine the curve division usage flag in the decoded bitstream. For example, if the curve division usage flag is 1 (indicating that it is used), it indicates that the current block uses curve division, and the index of the translation amplitude and / or curve shape transmitted in the bitstream can be decoded, wherein the decoding order of the translation amplitude and the index of the curve shape can be reversed, which is not limited in the embodiment of the present application. If the curve division usage flag is 0 (indicating that it is not used), the decoding of the curve division information is stopped.

[0062] In an embodiment of the present application, for the curve division mode, the decoder can indirectly determine it by decoding the translation amplitude in the code stream. If the translation amplitude is not 0 (indicating that the curve division mode is used), it means that curve division is used and the curve shape index continues to be decoded. If the translation amplitude is 0 (indicating that the curve division mode is not used), the decoding of the curve division information is stopped. Among them, there may be multiple combinations of the order of decoding the curve division information, the division mode of GPM, the information of the MV used for the two division parts of the current block, or other information related to GPM, which is not limited by the embodiment of the present application. The division mode of GPM, the information of the MV used for the two division parts of the current block, or the decoding method of other information related to GPM are not limited by this application.

[0063] It should be noted that, in an embodiment of the present application, a video image can be divided into multiple image blocks, each image block currently to be decoded can be called a decoding block, wherein each decoding block may include a first image component, a second image component and a third image component; and the current block is a decoding block in the video image that is currently to be predicted for the first image component, the second image component or the third image component.

[0064] Among them, assuming that the current block predicts the first image component, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block can also be called a luminance block; or, assuming that the current block predicts the second image component, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block can also be called a chrominance block.

[0065] It should also be noted that the prediction mode parameter indicates the encoding mode of the current block and parameters related to the mode.

[0066] In an embodiment of the present application, the division mode of the GPM may include: angle index information, and may also include: step index information and target division mode, etc., which is not limited by the embodiment of the present application.

[0067] It should be noted that the current GPM geometric division can divide any shape that can be calculated by a geometric function. Taking into account the application scenarios and implementation complexity, the existing geometric division methods are to divide a rectangular block into two blocks through a straight line. The geometric division here may be the division of coding units (CU coding unit) or the division of prediction units (PU prediction unit), which are collectively referred to as the current block. The divided blocks are triangular, rectangular or trapezoidal. The straight line that divides the rectangular block into two blocks can have different angles and different positions, so that a variety of combinations of triangles, rectangles and trapezoids can be divided. Usually, a transition is used for the edge area divided by the geometry, such as using a weighted average method to use a gradual coefficient transition on the divided edge, so that the divided edge is not too abrupt. The geometric division method currently used by VVC. It should be noted that in an embodiment of the present application, GPM is for the edge part of the object in the image, and divides the inter-frame block (i.e., the current block) into two non-rectangular sub-blocks, which are predicted separately and then weighted fused. For example, the non-rectangular form can be as follows Figure 4a-4i shown.

[0068] like Figure 5 As shown in the figure, VVC's geometric division divides the angle of a circle into 32 parts. The offset of the straight line relative to the center of the block can have 4 different values, one of which is 0, that is, the straight line passes through the center of the block. Currently, VVC uses some of the 32 angles, and the offset method varies with the angle. Some patterns that overlap with existing binary tree and ternary tree divisions are also eliminated.

[0069] In an embodiment of the present application, after the decoder obtains the GPM mode parameters and curve division information of the current block, the decoder can determine the first reference block and the second reference block corresponding to different partitions based on the GPM division mode and motion-related information in the GPM mode parameters, and then based on the GPM mode parameters and the curve division information, it can determine the first prediction parameters corresponding to the first reference block and the second prediction parameters corresponding to the second reference block.

[0070] In an embodiment of the present application, the first prediction parameter includes: a first weight value or first boundary position information; the second prediction parameter includes: a second weight value or second boundary position information.

[0071] It should be noted that the first weight value and the second weight value are weight values ​​corresponding to pixel points when GPM division is performed using the mask method, and the first boundary position information and the second boundary position information are boundary information of the first reference block and the second reference block.

[0072] In the embodiment of the present application, the decoder determines the first prediction parameter corresponding to the first reference block and the second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve division information. The first prediction parameter can be divided into two ways based on the difference in prediction parameters, as follows:

[0073] Method 1: The first prediction parameter includes: a first weight value; the second prediction parameter includes: a second weight value; the decoder determines a preset weight distribution corresponding to the current GPM mode according to the GPM mode parameter; determines the size information of the current block; updates the preset weight distribution according to the size information, the GPM mode parameter and the curve division information to obtain the curve weight distribution; obtains the first weight value and the second weight value according to the curve weight distribution.

[0074] It should be noted that the curve division information includes: translation amplitude and curve shape index; the decoder determines the offset direction according to the GPM mode parameters; determines the offset amount according to the size information, translation amplitude and curve shape index; according to the offset direction, the preset weight of each pixel position in the preset weight distribution is offset by the offset amount to obtain the updated curve weight of each pixel position, thereby obtaining the curve weight distribution.

[0075] Method 2: The first prediction parameter includes: first boundary position information; the second prediction parameter includes: second boundary position information; the decoder obtains the size information of the current block; based on the GPM mode parameter and the size information, the division boundary position information is determined; according to the GPM mode parameter, the size information and the curve division information, the division boundary position information is offset to determine the curve division boundary position information; based on the curve division boundary position information, the first boundary position information and the second boundary position information are determined.

[0076] It should be noted that the curve division information includes: translation amplitude and curve shape index; the decoder determines the offset direction according to the GPM mode parameters; determines the offset amount according to the size information, translation amplitude and curve shape index; according to the offset direction, the position of each pixel point of the division boundary position information is offset by the offset amount to obtain the curve division boundary position information.

[0077] In method 1, the preset weight distribution refers to the original mask corresponding to the straight line division when the GPM division mode is used, and the curve weight distribution is the weight distribution information corresponding to the curve division, which is updated by translating the preset weight distribution. The decoder mainly uses the size information, GPM mode parameters and curve division information to update the preset weight distribution and obtain the curve weight distribution. From the curve weight distribution, the first weight value corresponding to the first reference block of each pixel point of the current block and the second weight value corresponding to the second reference block are obtained.

[0078] In some embodiments of the present application, the decoder determines the offset direction based on the angle information in the GPM mode parameters, and determines the offset amount to be offset based on the size information, the translation amplitude and the curve shape index.

[0079] In method 2, the curve division method adopted by the decoder for the current block can use the pixel value of the first reference block as the first prediction value for part of the first reference block, and use the pixel value of the second reference block as the second prediction value for part of the second reference block. However, at the boundary position, the pixel points on the respective boundary information of the first reference block and the second boundary position information are offset to obtain the first boundary position information and the second boundary position information, and then the pixel points on the boundary are predicted based on the first boundary position information and the second boundary position information.

[0080] It should be noted that the GPM mode parameters are used to divide the current block. Based on the size information of the current block, the division boundary position information of the current block can be obtained; at this time, the decoder determines the offset direction according to the GPM mode parameters; determines the offset amount according to the size information, the translation amplitude and the curve shape index; according to the offset direction, the position of each pixel point of the division boundary position information is offset by the offset amount to obtain the curve division boundary position information.

[0081] In the embodiment of the present application, the decoder decodes the block partition information, and if the current block uses GPM, the GPM needs to decode the GPM partition mode.

[0082] For example, if the GPM uses a method of dividing CU or PU, the boundary of the division is determined according to the mode information and curve division information of the GPM. If the division boundary determined according to the mode information of the GPM is boundary, (x n ,y n ) is a point on the boundary. The decoder determines the boundary of the division as boundary' according to the GPM mode information and curve division information, (x′ n ,y′ n ) is a point on boundary'.

[0083] In view of the above implementation, in an embodiment of the present application, the decoder determines the offset direction according to the GPM mode parameters as follows: when the angle used by the GPM mode parameters represented by the GPM meets the horizontal judgment condition, the offset direction is determined to be the horizontal direction; when the angle used by the GPM mode parameters represented by the GPM meets the vertical judgment condition, the offset direction is determined to be the vertical direction.

[0084] In some embodiments of the present application, the decoder determines the offset direction according to the angle information in the GPM mode parameters.

[0085] In some embodiments of the present application, the vertical judgment condition is that the angle is within the horizontal angle range; the horizontal judgment condition is that the angle is within the vertical angle range, or other conditions except the horizontal judgment condition are vertical directions. The embodiments of the present application do not limit the judgment conditions, as long as the method can distinguish the moving direction.

[0086] For example, when the angle information in the GPM division parameter, that is, angle is 4 to 12 excluding 12 or angle is 20 to 28 excluding 28, the offset direction is the vertical direction, otherwise the offset direction is the horizontal direction, and the embodiment of the present application is not limited. That is, the vertical judgment condition can be If((angleIdx>=4&&angleIdx<12)||(angleIdx>=20&&angleIdx<28)).

[0087] In the following, for different offset modes, the decoder determines the description of the offset according to the size information, the translation amplitude and the curve shape index.

[0088] In the embodiment of the present application, the size information includes the length and width of the current block.

[0089] When the offset direction is horizontal, determine the first pixel position in the preset block in the non-offset direction at each pixel position according to each pixel position, the length and the number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determine the first curve information corresponding to the first pixel position according to the curve shape index; determine the first movement ratio based on the width and the number of preset offset points; determine the offset amount according to the first curve information, the translation amplitude and the first movement ratio, wherein the offset amount does not exceed the width of the current block.

[0090] It should be noted that the number of preset offset points is the number of points to be offset corresponding to different curve division modes. In the embodiment of the present application, the number of preset offset points can be 64, and the embodiment of the present application is not limited. The more preset offset points, the better the curve division.

[0091] In the embodiment of the present application, when the offset direction is a horizontal direction, the non-offset direction is a vertical direction; when the offset direction is a vertical direction, the non-offset direction is a horizontal direction.

[0092] In the embodiment of the present application, the decoder multiplies the first curve information with the translation amplitude and the first movement ratio to obtain an offset.

[0093] In some embodiments of the present application, the decoder determines the first pixel position in the preset block at each pixel position in the non-offset direction based on each pixel position, the length, and the number of preset offset points as follows: determining a first scaling ratio based on each pixel position, the length, and the number of preset offset points; determining a first sub-offset based on the length and the number of preset offset points; and determining the first pixel position in the preset block at each pixel position in the non-offset direction based on the first scaling ratio and the first sub-offset.

[0094] In the embodiment of the present application, the decoder multiplies the ordinate of each pixel position by the ratio of the number of preset offset points to the length to obtain a first scaling ratio, or the decoder uses the logarithm of the length and the number of preset offset points with a base of 2 and a shift to obtain the first scaling ratio; the decoder then obtains the first sub-offset according to the number of preset offset points and (2*length); finally, the decoder adds the first scaling ratio and the first sub-offset to obtain the first pixel position. The number of preset offset points may be 64, 32, 16, 8, etc., which is not limited in the embodiment of the present application.

[0095] For example, it is assumed that the decoder obtains mask P' (curve weight distribution) according to the translation amplitude and curve shape used for curve division and mask P (preset weight distribution), and the translation amplitude is set to curveAmplitude, the index of the curve shape used for translation is curveIdx, there are NUM_CURVE curve shapes in total, and LUT (curve information) is curveFactor[NUM_CURVE][N], where N is equal. Let the width of the current block be width, the length be height, and the angle used by GPM be angle.

[0096] If the current block uses the horizontal direction, P' of each pixel (x, y) is as shown in formula (1), as follows:

[0097] P'[x][y]= P[x-offsetX][y] (1)

[0098] Wherein, offsetX is the offset. The offset is obtained by formula (2), as follows:

[0099] offsetX= curveFactor[curveIdx][y N ]* curveAmplitude *Scale1 (2)

[0100] Here, curveFactor[curveIdx][y N ] is the first curve information, curveAmplitude is the translation amplitude, and Scale1 is the first movement ratio.

[0101] Here, the first moving ratio is obtained by formula (3), y N The following is obtained for formula (4):

[0102] Scale1=width / N (3)

[0103] y N =y*N / height+offset1 (4)

[0104] Wherein, offset1 is the first sub-offset, which is determined by formula (5) as follows:

[0105] Offset1=N / (2*height) (5)

[0106] It should be noted that x-offsetX will not exceed the normal range of the current block, such as 0 to width-1.

[0107] In some embodiments of the present application, when the offset direction is a vertical direction, the decoder determines a second pixel position in a preset block in a non-offset direction at each pixel position based on each pixel position, a width, and a number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determines second curve information corresponding to the second pixel position based on a curve shape index; determines a second movement ratio based on a length and the number of preset offset points; determines an offset amount based on the second curve information, a translation amplitude, and a second movement ratio, wherein the offset amount does not exceed the length of the current block.

[0108] In the embodiment of the present application, the decoder multiplies the second curve information with the translation amplitude and the second movement ratio to obtain an offset.

[0109] In some embodiments of the present application, the decoder determines the second pixel position located in the preset block in the non-offset direction at each pixel position based on each pixel position, the width, and the number of preset offset points as follows: the decoder determines the second scaling ratio based on each pixel position, the width, and the number of preset offset points; determines the second sub-offset based on the width and the number of preset offset points; and determines the second pixel position located in the preset block in the non-offset direction at each pixel position based on the second scaling ratio and the second sub-offset.

[0110] In an embodiment of the present application, the decoder multiplies the horizontal coordinate of each pixel position by the ratio of the number of preset offset points to the width to obtain a second scaling ratio, or the decoder uses the width and the number of preset offset points with the logarithm to the base 2 and the shift to obtain the second scaling ratio; the decoder then obtains the second sub-offset based on the number of preset offset points and (2*width); finally, the decoder adds the second scaling ratio and the second sub-offset to obtain the second pixel position.

[0111] For example, it is assumed that the decoder obtains mask P' (curve weight distribution) according to the translation amplitude and curve shape used for curve division and mask P (preset weight distribution), and the translation amplitude is set to curveAmplitude, the index of the curve shape used for translation is curveIdx, there are NUM_CURVE curve shapes in total, and LUT (curve information) is curveFactor[NUM_CURVE][N], where N is equal. Let the width of the current block be width, the length be height, and the angle used by GPM be angle.

[0112] If the current block uses the vertical direction, P' of each pixel (x, y) is as shown in formula (6):

[0113] P'[x][y]= P[x][y-offsetY] (6)

[0114] Wherein, offsetY is the offset. The offset is obtained by formula (7), as follows:

[0115] offsetX= curveFactor[curveIdx][x N ]* curveAmplitude *Scale2 (7)

[0116] Here, curveFactor[curveIdx][x N ] is the second curve information, curveAmplitude is the translation amplitude, and Scale2 is the second movement ratio.

[0117] Here, the second moving ratio is obtained by formula (8), x N The following is obtained for formula (9):

[0118] Scale2= height / N (8)

[0119] x N =x*N / width+offset2 (9)

[0120] Wherein, offset2 is the second sub-offset, which is determined by formula (10) as follows:

[0121] offset2=N / (2*width) (10)

[0122] It should be noted that y-offsetY will not exceed the normal range of the current block, such as 0 to height-1.

[0123] Based on the above two offset directions, another way to express the offset process is:

[0124] If((angleIdx>=4&&angleIdx<12)||(angleIdx>=20&&angleIdx<28))

[0125] offsetCX=0

[0126] offsetCY=curveFactor[curveIdx][xL*N / nW+N / 2nW]*curveAmplitude*nH / N

[0127] Otherwise(shiftHor is equal to 1), the following applies:

[0128] offsetCX=curveFactor[curveIdx][yL*N / nH+N / 2nH]*curveAmplitude*nW / N

[0129] offsetCY=0

[0130] Among them, nW is the width, nH is the length, offsetCX is offsetX, offsetCY is offsetY, and angleIdx is the angle index.

[0131] In some embodiments of the present application, the decoder may first determine the position (x', y') of each pixel point (x, y) of the current block on the preset weight distribution according to the translation amplitude and curve shape used for curve division, and then calculate predSamples[x][y] for each pixel point (x, y) based on the preset weight distribution, the offset direction and the offset amount to temporarily calculate the required P[x'][y'].

[0132] For example, if the current block uses the horizontal direction, x'=x-offsetX, y'=y

[0133] If the current block uses the vertical direction, x'=x, y'=y-offsetY

[0134] Where offsetX and offsetY are obtained in the same way as above.

[0135] It should be noted that in the embodiment of the present application, the decoder may first calculate the mask P, and then calculate predSamples[x][y] for each point (x, y). It is also possible to temporarily calculate the required P[x][y] when calculating predSamples[x][y] for each point (x, y). In this case, the curve partitioning method is used to temporarily calculate the required P[x'][y'] when calculating predSamples[x][y] for each point (x, y). The embodiment of the present application does not limit this, and the final purpose is the same.

[0136] After the decoder obtains the first weight value and the second weight value using method 1, the decoder can determine the prediction value of the current block based on the first weight value, the first pixel value of the first reference block, the second weight value and the second pixel value of the second reference block.

[0137] In an embodiment of the present application, the decoder performs a weighted sum operation on the first weight value, the first pixel value of the first reference block, the second weight value, and the second pixel value of the second reference block to obtain a prediction value of the current block.

[0138] Exemplarily, let the pixel value of the first reference block at the position (x, y) be predSamplesLA[x][y], and the pixel value of the second reference block at the position (x, y) be predSamplesLB[x][y], the decoder obtains the mask P mask matrix according to the division mode of GPM, each pixel position (x, y) on the mask has a value P[x][y] marking the weight of the first reference block (x, y), M is the sum of the weights of the corresponding positions of the two reference blocks, then the weight of the second reference block (x, y) is MP[x][y]. The decoder obtains the mask P' according to the translation amplitude and curve shape used for curve division and the mask P.

[0139] That is, the predicted value of a pixel point (x, y) of the current block is shown in formula (11):

[0140] predSamples[x][y]=(P'[x][y]*predSamplesLA[x][y]+(M-P'[x][y])*predSamplesLB

[0141] [x][y]) / M(11)

[0142] Among them, predSamples[x][y] is the predicted value of a pixel point (x, y) in the current block.

[0143] It should be noted that M can be a power of 2 integer, such as 4, 8, 16, etc., so that the division in the equation can be written as a right shift. If predSamples[x][y] exceeds the normal value range, it needs to be processed to be within the value range, and this application does not limit it here.

[0144] After the decoder obtains the first boundary position information and the second boundary position information by adopting the second method, the decoder determines the prediction value of the current block based on the first prediction parameter and the second prediction parameter as follows: the decoder divides the current block according to the first boundary position information and the second boundary position information to obtain the first block and the second block; predicts the first block based on the first pixel value of the first reference block, and predicts the second block based on the second pixel value of the second reference block, thereby determining the prediction value of the current block.

[0145] It can be understood that the decoder predicts the current block by using curve division in the GPM mode so that the divided boundary is more closely aligned with the edge of the object or the edge of two parts with different motion vectors, thereby improving the accuracy of encoding and decoding.

[0146] It should be noted that after obtaining the prediction block of the current block, the decoder decodes the residual information of the current block, including the decoding coefficients, dequantizes and inversely transforms to obtain the residual block, and adds the residual block to the prediction block to obtain the decoded block of the current block.

[0147] The inter-frame prediction provided in the embodiment of the present application adopts a curve division method.

[0148] In an embodiment of the present application, the curve division information is a division method in curve division; curve division is to translate the pixel points on the direct division line in the preset straight line geometric division to obtain a continuous curve division line to realize geometric division.

[0149] In some embodiments of the present application, each pixel point of a straight line dividing line is continuously translated according to different moving distances and the same moving direction to obtain a curved line dividing line; or, each pixel point of a straight line dividing line is continuously translated according to different moving vectors to obtain a curved line dividing line.

[0150] That is to say, if the points on a straight line segment (the straight lines mentioned here are all line segments) are translated in the same direction, the translation distance of each point is not the same, the distances of adjacent points are not much different, and the continuity of the line is maintained, a curved segment can be obtained. Or if the points on a straight line segment are translated, the translation vectors of each point are not the same, the vectors of adjacent points are not much different, and the continuity of the line is maintained, a curved segment can also be obtained. For video encoding and decoding, the so-called straight lines are composed of points whose pixel positions are on the same straight line or approximately on the same straight line. If these points are translated in the same direction, the translation distances of each point are not the same, and the distances of adjacent points meet certain requirements, a curved segment can be obtained. If these points are translated, the translation vectors of each point are not the same, and the distances of adjacent points meet certain requirements, a curved segment can be obtained.

[0151] For example, Figure 6 As shown in the diagram of the transformation from straight line to curve, the points on the boundary line of the two blocks in the geometric partition are translated as shown by the arrows in the figure above, and the boundary line can be transformed into a curve segment, and the straight line partition can be transformed into a curve partition. The points on the transition area near the dividing line are translated as described above, and the transition area of ​​the straight line segment can be transformed into the transition area of ​​the curve segment. For the method of geometric partitioning realized by mask, the values ​​of the points on the mask are translated as described above, and the mask of the straight line geometric partition can be transformed into the mask of the curve geometric partition.

[0152] In some embodiments of the present application, the moving distance is the distance between the corresponding pixel position on the preset curve shape and the straight line dividing line; the moving vector is a preset vector corresponding to the pixel position on the preset curve shape.

[0153] It should be noted that, in the embodiment of the present application, a LUT (look up table) may be used to store preset vectors or preset distances corresponding to curve divisions.

[0154] In an embodiment of the present application, it is assumed that all points move in the same direction (horizontally, vertically, or in the direction of another angle), and the LUT stores the moving distances of several points for a curve shape, and one or more curve shapes can be stored in the LUT. Among them, the curve shape is the shape of the curve itself. When a straight line is translated in a certain direction according to the direction of the curve, the curve may not be the same as the shape of the curve itself. If the divided line segment is to be translated according to a certain curve shape, each point on the line segment finds the corresponding position in the corresponding curve shape in the LUT, and moves in a given direction according to the indicated moving distance, and the point can be restricted from moving beyond a certain boundary. If the mask divided by a straight line is to be changed into a mask divided by a curve, after determining the direction of the translation, the values ​​of different points on the mask perpendicular to the translation direction are moved according to the moving distances of the corresponding positions on the LUT, and the values ​​of the points on the mask with the same translation direction are moved according to the same moving distance. It can be understood that the new mask divided by the curve is obtained by referring to the mask divided by the straight line.

[0155] In some embodiments of the present application, the values ​​of points beyond the required range are discarded, and the values ​​of points can be restricted to not exceed a certain boundary after being moved, and the values ​​of missing points are supplemented according to the original values ​​at the edge of the valid range.

[0156] In some embodiments of the present application, the value of a point in a mask divided by a curve is found from the value of a corresponding point in a mask divided by a straight line in the opposite direction and distance to which it should move. If the point to be found exceeds the valid limit, it is replaced by the value of a point on the limit on the same straight line as the moving direction.

[0157] In some embodiments of the present application, assuming that not all points move in the same direction, the LUT stores the movement vectors of several points for a curve shape, and one or more curve shapes can be stored in the LUT. If the divided line segment is to be translated according to a certain curve shape, each point on the line segment finds the corresponding position in the corresponding curve shape in the LUT and moves according to the indicated movement vector. Due to the discontinuity caused by the movement, it may be necessary to supplement points at the discontinuous position to maintain the continuity of the curve. If the mask divided by the straight line is to be changed into the mask divided by the curve, after determining the direction of the translation, the values ​​of different points in a certain direction on the mask can be moved according to the movement vector of the corresponding position on the LUT, and the values ​​of the points are supplemented for the places causing discontinuity. The values ​​of the points on the mask in the same vertical direction as the above direction are moved according to the same movement vector. The values ​​of the points are supplemented for the places causing discontinuity. The values ​​of the points beyond the required range are discarded, and the values ​​of the points can also be restricted to not exceed a certain boundary after the movement. The values ​​of the missing points are supplemented according to the original values ​​at the edge of the required range.

[0158] In some embodiments of the present application, the moving direction is formed by a combination of a horizontal direction and a vertical direction; or, the moving direction is related to a division direction of the GPM.

[0159] In some embodiments of the present application, when the moving distance is not an integer pixel, the sub-pixel position is interpolated or rounded; or, the moving distance is positively correlated with the length of the current block; or, the moving distance is positively correlated with the smallest value of the length and width of the current block.

[0160] In some embodiments of the present application, the moving direction may be horizontal or vertical, or any other direction. Assuming that the moving direction is not horizontal or vertical, it can be divided into horizontal and vertical movement.

[0161] For example, a 2-pixel movement in a 45° direction can be split into a √2-pixel movement in a horizontal 0° direction plus a √2-pixel movement in a vertical 90° direction.

[0162] In the embodiment of the present application, when the moving distance is not an integer pixel, one possible implementation method is to take an approximate integer pixel position, such as a rounding operation, and another possible implementation method is to interpolate the sub-pixel position.

[0163] Exemplarily, √2 pixel distances can be approximated as 1 pixel distance or 2 pixel distances, or 1.5 pixel distances or 1.4 pixel distances. The values ​​of 1.5 pixel distances or 1.4 pixel distances need to be interpolated. Other angles can also be split into horizontal and vertical movements according to trigonometric functions or pre-calculated ratios. The horizontal movement direction can be represented by a positive value for the 0° direction and a negative value for the 180° direction, and the vertical movement direction can be represented by a positive value for the 90° direction and a negative value for the 270° direction.

[0164] In the embodiment of the present application, one possible implementation is to use only the horizontal movement direction plus the vertical movement direction; one possible implementation is to use only the horizontal movement direction plus the vertical movement direction plus the 45°, 135°, 225°, 315° movement directions. Among them, the 45°, 135°, 225°, 315° movement directions can be decomposed into horizontal and vertical movement directions with equal values, and the four angles can be represented by the positive and negative signs of the horizontal and vertical directions; one possible implementation is to use multiple angle directions.

[0165] It should be noted that the direction of movement can be related to the division direction of GPM. In the case of only horizontal and vertical movement, if the straight line angle used for GPM straight line division is horizontal or close to horizontal, the curve division uses the vertical movement direction; if the straight line angle used for GPM straight line division is vertical or close to vertical, the curve division uses the horizontal movement direction. More generally, some thresholds can be set to use the horizontal movement direction within a certain threshold range when the straight line angle or the angle used for division or the division mode is within a certain threshold range, and use the vertical movement direction within another range. The same principle applies, when only the horizontal movement direction plus the vertical movement direction plus 45°, 135°, 225°, 315° movement directions are used, or when multiple movement directions are used, a threshold range is set for the straight line angle or the angle used for division or the division mode to distinguish the usage range of each movement direction.

[0166] It should be noted that the moving distance may be related to the length and width of the current block. One possible implementation is that the moving distance is proportional to the length of the current block in the same direction as the current moving direction. For example, if the length of the current block in the same direction as the current moving direction is 64, the distance moved by all points in the current block is the value found in the LUT. If the length of the current block in the same direction as the current moving direction is 32, the distance moved by all points in the current block is 1 / 2 of the value found in the LUT. One possible implementation is that the moving distance is proportional to the shorter length of the current block.

[0167] In the embodiment of the present application, different translation amplitudes can be set, i.e., 1 / 4 times, 1 / 2 times, 1 times, 2 times, etc. If the current block selects 1 / 4 times, the distance moved by all points in the current block is 1 / 4 times the moving distance found in the LUT. The translation amplitude can be 0 times, i.e., the moving distance of all points in the current block is 0, in which case, the curve division does not actually work. A negative translation amplitude indicates that the moving direction is executed in the opposite direction of the original direction.

[0168] That is to say, the translation amplitude is a multiple of the actual moving distance or the moving vector, or a geometric relationship.

[0169] In some embodiments of the present application, not all points to be offset may be translated in the same direction. Points on a straight line segment may be translated, and the translation vectors of each point are not the same, and the translation vectors of adjacent points are not very different. The continuity of the line is maintained, and a curved line segment can also be obtained.

[0170] For video encoding and decoding, the so-called straight line is composed of points whose pixel positions are on the same straight line or approximately on the same straight line. These points are translated in the same direction, and the translation distances of each point are not the same, and the distances of adjacent points meet certain requirements, and a curve segment can be obtained. These points are translated, and the translation vectors of each point are not the same, and the distances of adjacent points meet certain requirements, and a curve segment can be obtained.

[0171] Assuming that not all points move in the same direction, the LUT stores the movement vectors of several points for a curve shape (the LUT can store information related to one or more curve shapes). If the divided line segment is to be translated according to a certain curve shape, each point on the line segment finds the corresponding position in the corresponding curve shape in the LUT and moves according to the indicated movement vector. Due to the discontinuity caused by the movement, it may be necessary to add points at the discontinuous position to maintain the continuity of the curve. If the mask divided by a straight line is to be converted into a mask divided by a curve, after determining the direction of the translation, the values ​​of different points in a certain direction on the mask can be moved according to the movement vector of the corresponding position on the LUT, and the values ​​of the points that cause discontinuity are added. The values ​​of the points on the mask in the same vertical direction as the above direction are moved according to the same movement vector. The values ​​of the points that cause discontinuity are added. The values ​​of the points that exceed the required range are discarded, and the values ​​of the points can also be restricted to not exceed a certain boundary after movement, and the values ​​of the missing points are added according to the original values ​​at the edge of the required range.

[0172] In the embodiment of the present application, a LUT is used to determine the translation distance or the translation vector.

[0173] It can be understood that by using the translation method to achieve curve geometric division, the division boundaries can be made more flexible and more closely fit the edges of objects or the edges of two parts with different motion vectors, thereby improving the encoding performance. In addition, the translation method has low complexity and is easy to implement in achieving curve geometric division.

[0174] The embodiment of the present application provides an inter-frame prediction method, which is applied to a video encoding device, i.e., an encoder. The function implemented by the method can be implemented by calling a program code by a second processor in the video encoding device, and of course the program code can be stored in a second memory. It can be seen that the video encoding device at least includes a second processor and a second memory.

[0175] Figure 7 FIG. 1 is a schematic diagram of an implementation flow of an inter-frame prediction method according to an embodiment of the present application. Figure 7 As shown, the method includes:

[0176] S201. Determine prediction mode parameters of the current block.

[0177] S202: When the prediction mode parameter indicates that the inter-frame prediction of the current block is determined using the inter-frame geometric partitioning prediction mode GPM, determine the GPM mode parameters and curve partitioning information of the current block.

[0178] S203. Determine a first reference block and a second reference block based on GPM mode parameters.

[0179] S204. Determine a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve division information.

[0180] S205. Determine a prediction value of the current block based on the first prediction parameter and the second prediction parameter.

[0181] In an embodiment of the present application, a video image can be divided into multiple image blocks, each image block currently to be encoded can be called a coding block (Coding Block, CB), wherein each coding block may include a first image component, a second image component, and a third image component; and the current block is a coding block in the video image that is currently to be predicted for the first image component, the second image component, or the third image component.

[0182] Among them, assuming that the current block predicts the first image component, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block can also be called a luminance block; or, assuming that the current block predicts the second image component, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block can also be called a chrominance block.

[0183] In some embodiments of the present application, the encoder determines the prediction mode parameters of the current block as follows: the encoder determines the image components to be predicted of the current block; based on the parameters of the current block, the image components to be predicted are respectively predicted and encoded using multiple prediction modes, and the rate-distortion cost results corresponding to each prediction mode under the multiple prediction modes are calculated; the minimum rate-distortion cost result is selected from the multiple rate-distortion cost results calculated, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block.

[0184] It should be noted that the prediction mode parameters indicate the coding mode of the current block and parameters related to the mode. Usually, the prediction mode parameters of the current block can be determined by using rate distortion optimization (RDO).

[0185] In some embodiments of the present application, the encoder determines the prediction mode parameters of the current block as follows: the encoder determines the image components to be predicted of the current block; based on the parameters of the current block, the image components to be predicted are respectively predicted and encoded using multiple prediction modes, and the rate-distortion cost results corresponding to each prediction mode under the multiple prediction modes are calculated; the minimum rate-distortion cost result is selected from the multiple rate-distortion cost results calculated, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block.

[0186] That is to say, on the encoder side, multiple prediction modes can be used for the current block to encode the predicted image components respectively. Here, the multiple prediction modes generally include inter-frame prediction mode, traditional intra-frame prediction mode and non-traditional intra-frame prediction mode, and the traditional intra-frame prediction mode can include direct current (DC) mode, plane (PLANAR) mode and angle mode, etc., and the non-traditional intra-frame prediction mode can include MIP mode, cross-component linear model prediction (CCLM) mode, intra-frame block copy (IBC) mode and PLT (Palette) mode, etc. The inter-frame prediction mode may include: inter-frame geometric partition prediction mode (GPM), triangle prediction mode (TPM), etc. In the embodiment of the present application, a curve partition mode is also included, wherein, in the embodiment of the present application, all curve partition information of the curve partition is stored in the LUT.

[0187] In this way, after encoding the current block respectively using multiple prediction modes, the rate-distortion cost result corresponding to each prediction mode can be obtained; then the minimum rate-distortion cost result is selected from the multiple rate-distortion cost results obtained, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block; in this way, the current block can finally be encoded using the determined prediction mode, and in this prediction mode, the prediction residual can be made small, which can improve the coding efficiency.

[0188] In the embodiment of the present application, since there can be many kinds of prediction modes, and there can be many kinds of specific division modes in one prediction mode, it is necessary to first traverse all prediction modes, and then traverse the division modes for each prediction mode, etc. The multiple prediction modes in the embodiment of the present application include the GPM mode, and the present application is implemented based on the scenario of the GPM prediction mode.

[0189] In an embodiment of the present application, when multiple prediction modes include a GPM mode, the process by which the encoder calculates the rate-distortion cost result corresponding to the GPM prediction mode can be: the encoder obtains a set of motion-related information corresponding to the current block; uses each GPM partition mode in the GPM partition mode to traverse each motion-related information in the motion-related information set, and then uses the curve partition mode to traverse each GPM mode to obtain the rate-distortion cost result corresponding to the GPM prediction mode.

[0190] It should be noted that each motion-related information may be two MVs, which is not limited in the embodiment of the present application. The motion-related information is stored in a merge list.

[0191] In an embodiment of the present application, if the encoder uses a mask method when performing GPM division, the encoder will determine whether the current block uses GPM and curve division. One possible method is to traverse all possible mv combinations (motion-related information sets) for the current block, traverse all possible GPM division modes for each mv combination, and traverse all curve division modes for each GPM division mode, including all translation amplitudes and curve shapes. For each possible situation, the actual coding cost (rate-distortion cost result) is obtained, and the GPM division mode and curve division mode corresponding to the best coding cost (i.e., the minimum rate-distortion cost result) are taken as the GPM mode and curve division mode of the current block.

[0192] In an embodiment of the present application, for each of the above possible situations, cost=SAD+bitcost or cost=SATD+bitcost is used as the encoding cost, where SAD is the Sum of Absolute Difference, SATD is the Sum of Absolute Transformed Difference, and bitcost is the cost of the encoding flag bit.

[0193] In some embodiments of the present application, when multiple prediction modes include a GPM mode, the process by which the encoder calculates the rate-distortion cost result corresponding to the GPM prediction mode may be: the encoder obtains a set of motion-related information corresponding to the current block; using the GPM partition mode, traverses each motion-related information in the motion-related information set to obtain a first rate-distortion cost result corresponding to the GPM partition mode; for at least one target GPM partition mode with the lowest first rate-distortion cost result, traverses the curve partition mode to obtain a rate-distortion cost result corresponding to the GPM prediction mode.

[0194] In an embodiment of the present application, another possible method is that the encoder traverses all possible mv combinations for the current block, traverses all possible GPM division modes for each mv combination, and uses cost=SAD+bitcost or cost=SATD+bitcost as the encoding cost for each possible situation. Take the GPM division modes with the best encoding costs mentioned above, and traverse all curve division modes for each GPM division mode, including all translation amplitudes and curve shapes. For each possible situation, use cost=SAD+bitcost or cost=SATD+bitcost as the encoding cost. Take the GPM division mode and curve division mode with the best encoding cost as the GPM mode of the current block and the curve division mode of the current block.

[0195] It should be noted that the implementation principle of each GPM division mode traversing all curve division modes is consistent with the principle of using curve division information for offset on the decoding side, which will not be repeated here.

[0196] In the embodiment of the present application, the encoder determines the best encoding cost, that is, the GPM mode parameter and curve division information of the current block corresponding to the rate-distortion cost result, and the encoder can determine the first reference block and the second reference block corresponding to the motion-related information based on the GPM mode parameter, and then determine the first prediction parameter corresponding to the first reference block and the second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve division information. Finally, the encoder determines the prediction value of the current block based on the first prediction parameter and the second prediction parameter.

[0197] In an embodiment of the present application, the first prediction parameter includes: a first weight value or first boundary position information; the second prediction parameter includes: a second weight value or second boundary position information.

[0198] It should be noted that the first weight value and the second weight value are weight values ​​corresponding to pixel points when GPM division is performed using the mask method, and the first boundary position information and the second boundary position information are boundary information of the first reference block and the second reference block.

[0199] In the embodiment of the present application, the encoder determines the first prediction parameter corresponding to the first reference block and the second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve division information. The determination can be divided into two ways based on the difference in prediction parameters, as follows:

[0200] Method 1: The first prediction parameter includes: a first weight value; the second prediction parameter includes: a second weight value; the encoder determines a preset weight distribution corresponding to the current GPM mode based on the GPM mode parameter; determines the size information of the current block; updates the preset weight distribution based on the size information, GPM mode parameters and curve division information to obtain a curve weight distribution; obtains the first weight value and the second weight value based on the curve weight distribution.

[0201] It should be noted that the curve division information includes: translation amplitude and curve shape index; the encoder determines the offset direction according to the GPM mode parameters; determines the offset amount according to the size information, translation amplitude and curve shape index; according to the offset direction, the preset weight of each pixel position in the preset weight distribution is offset by the offset amount to obtain the updated curve weight of each pixel position, thereby obtaining the curve weight distribution.

[0202] Method 2: The first prediction parameter includes: first boundary position information; the second prediction parameter includes: second boundary position information; the encoder obtains the size information of the current block; based on the GPM mode parameter and the size information, the division boundary position information is determined; according to the GPM mode parameter, the size information and the curve division information, the division boundary position information is offset to determine the curve division boundary position information; based on the curve division boundary position information, the first boundary position information and the second boundary position information are determined.

[0203] It should be noted that the curve division information includes: translation amplitude and curve shape index; the encoder determines the offset direction according to the GPM mode parameters; determines the offset amount according to the size information, translation amplitude and curve shape index; according to the offset direction, the position of each pixel point of the division boundary position information is offset by the offset amount to obtain the curve division boundary position information.

[0204] In method 1, the preset weight distribution refers to the original mask corresponding to the straight line division when the GPM division mode is used, and the curve weight distribution is the weight distribution information corresponding to the curve division, which is updated by translating the preset weight distribution. The encoder mainly uses the size information, GPM mode parameters and curve division information to update the preset weight distribution and obtain the curve weight distribution. From the curve weight distribution, the first weight value corresponding to the first reference block of each pixel point of the current block and the second weight value corresponding to the second reference block are obtained.

[0205] In some embodiments of the present application, the encoder determines the offset direction based on the angle information in the GPM mode parameters, and determines the offset amount to be offset based on the size information, the translation amplitude and the curve shape index.

[0206] In method 2, the encoder adopts a curve division method for the current block, and can use the pixel value of the first reference block as the first prediction value for part of the first reference block, and use the pixel value of the second reference block as the second prediction value for part of the second reference block. However, at the boundary position, the pixel points on the respective boundary information of the first reference block and the second boundary position information are offset to obtain the first boundary position information and the second boundary position information, and then the pixel points on the boundary are predicted based on the first boundary position information and the second boundary position information.

[0207] It should be noted that the encoder uses the GPM mode parameters to divide the current block, and based on the size information of the current block, the division boundary position information of the current block can be obtained; at this time, the encoder determines the offset direction according to the GPM mode parameters; determines the offset amount according to the size information, translation amplitude and curve shape index; according to the offset direction, the position of each pixel point of the division boundary position information is offset by the offset amount to obtain the curve division boundary position information.

[0208] In view of the above implementation, in an embodiment of the present application, the encoder determines the offset direction according to the GPM mode parameters as follows: when the angle used by the GPM mode parameters represented by the GPM meets the horizontal judgment condition, the offset direction is determined to be the horizontal direction; when the angle used by the GPM mode parameters represented by the GPM meets the vertical judgment condition, the offset direction is determined to be the vertical direction.

[0209] In the following, for different offset modes, the encoder determines the description of the offset based on the size information, translation amplitude and curve shape index.

[0210] In the embodiment of the present application, the size information includes the length and width of the current block.

[0211] When the offset direction is horizontal, the encoder determines the first pixel position in the preset block in the non-offset direction at each pixel position according to each pixel position, the length and the number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determines the first curve information corresponding to the first pixel position according to the curve shape index; determines the first movement ratio based on the width and the number of preset offset points; determines the offset according to the first curve information, the translation amplitude and the first movement ratio, wherein the offset does not exceed the width of the current block.

[0212] In some embodiments of the present application, the encoder determines the first pixel position in the preset block at each pixel position in a non-offset direction based on each pixel position, length, and the number of preset offset points as follows: determining a first scaling ratio based on each pixel position, length, and the number of preset offset points; determining a first sub-offset based on the length and the number of preset offset points; and determining the first pixel position in the preset block at each pixel position in the non-offset direction based on the first scaling ratio and the first sub-offset.

[0213] In some embodiments of the present application, when the offset direction is a vertical direction, the encoder determines a second pixel position in a preset block in a non-offset direction at each pixel position based on each pixel position, the width, and the number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determines second curve information corresponding to the second pixel position based on a curve shape index; determines a second movement ratio based on the length and the number of preset offset points; determines an offset amount based on the second curve information, the translation amplitude, and the second movement ratio, wherein the offset amount does not exceed the length of the current block.

[0214] In the embodiment of the present application, the encoder multiplies the second curve information with the translation amplitude and the second movement ratio to obtain an offset.

[0215] In some embodiments of the present application, the encoder determines the second pixel position located in the preset block in the non-offset direction at each pixel position based on each pixel position, the width, and the number of preset offset points as follows: the encoder determines the second scaling ratio based on each pixel position, the width, and the number of preset offset points; determines the second sub-offset based on the width and the number of preset offset points; and determines the second pixel position located in the preset block in the non-offset direction at each pixel position based on the second scaling ratio and the second sub-offset.

[0216] Furthermore, after the encoder obtains the first weight value and the second weight value using method one, the encoder can determine the prediction value of the current block based on the first weight value, the first pixel value of the first reference block, the second weight value and the second pixel value of the second reference block.

[0217] In an embodiment of the present application, the encoder weights and averages the first weight value, the first pixel value of the first reference block, the second weight value, and the second pixel value of the second reference block to obtain a prediction value of the current block.

[0218] After the encoder obtains the first boundary position information and the second boundary position information by adopting the second method, the encoder divides the current block according to the first boundary position information and the second boundary position information to obtain the first block and the second block; predicts the first block based on the first pixel value of the first reference block, and predicts the second block based on the second pixel value of the second reference block, thereby determining the predicted value of the current block.

[0219] The inter-frame prediction provided in the embodiment of the present application adopts a curve division method.

[0220] In an embodiment of the present application, the curve division information is a division method in curve division; curve division is to translate the pixel points on the direct division line in the preset straight line geometric division to obtain a continuous curve division line to realize geometric division.

[0221] In some embodiments of the present application, each pixel point of a straight line dividing line is continuously translated according to different moving distances and the same moving direction to obtain a curved line dividing line; or, each pixel point of a straight line dividing line is continuously translated according to different moving vectors to obtain a curved line dividing line.

[0222] In some embodiments of the present application, the moving distance is the distance between the corresponding pixel position on the preset curve shape and the straight line dividing line; the moving vector is a preset vector corresponding to the pixel position on the preset curve shape.

[0223] In some embodiments of the present application, the moving direction is formed by a combination of a horizontal direction and a vertical direction; or, the moving direction is related to a division direction of the GPM.

[0224] In some embodiments of the present application, when the moving distance is not an integer pixel, the sub-pixel position is interpolated or rounded; or, the moving distance is positively correlated with the length of the current block; or, the moving distance is positively correlated with the smallest value of the length and width of the current block.

[0225] It should be noted that the principle of the prediction process of the current block of the encoder is consistent with that of the decoder side, and the same explanation and description will not be repeated here.

[0226] In some embodiments of the present application, if the encoder uses the CU or PU division method when performing GPM division, the encoder needs to determine whether the current block uses the GPM division mode (or GPM prediction mode) and the curve division mode.

[0227] In an embodiment of the present application, the encoder may traverse all GPM partitioning modes and curve partitioning modes, and for each possible situation, encode the divided CU or PU respectively. This process may involve the process of iteratively encoding the CU or PU of the next layer to obtain the encoding cost. The encoder tries other methods that do not use GPM and curve partitioning, and determines the prediction mode selected for the current block with the mode with the lowest encoding cost.

[0228] It can be understood that the encoder predicts the current block by using curve division in the GPM mode, so that the divided boundary is more closely aligned with the edge of the object or the edge of two parts with different motion vectors, thereby improving the encoding accuracy.

[0229] In some embodiments of the present application, after S202, as Figure 8 As shown, an inter-frame prediction method provided by the present application also includes: S206-S207.

[0230] S206: When the prediction mode corresponding to the minimum rate distortion cost result is the curve division mode of the GPM prediction mode, a curve division use flag is generated according to the curve division information, and the curve division information is encoded.

[0231] S207: The curve division usage flag and the encoded curve division information are input into the bitstream.

[0232] In the embodiment of the present application, after the encoder determines the minimum rate distortion cost result, if the corresponding prediction mode of the minimum rate distortion cost result is the curve division mode under the GPM prediction mode, the encoder can generate a curve division use flag according to the curve division information, and encode the curve division information. Finally, the encoder writes the curve division use flag and the encoded curve division information into the bitstream for use by the decoder during decoding. However, in the present application, it is also possible to only write the encoded curve division information into the bitstream for use by the decoder during decoding, which is not limited by the embodiment of the present application.

[0233] In some embodiments of the present application, the curve division information includes: a translation amplitude and a curve shape index; the encoder can generate a curve division use flag according to the translation amplitude, and when the curve division use flag indicates that it is used, encode the translation amplitude and the curve shape index to obtain the encoded curve division information. When the curve division use flag indicates that it is not used, stop encoding the translation amplitude and the curve shape index.

[0234] In some embodiments of the present application, the encoder may also encode the translation amplitude to obtain the encoded translation amplitude; when the translation amplitude is represented by using curve division, the curve shape index is encoded to obtain the encoded curve shape index, wherein the encoded translation amplitude and the encoded curve shape index are the encoded curve division information; when the translation amplitude is represented by using curve division and the curve division is not used, the encoding of the curve shape index is stopped.

[0235] It should be noted that in the embodiment of the present application, the encoder can use the GPM use flag or other flags to derive whether the GPM is used. If the current block uses the GPM mode, the information of the selected GPM division mode is encoded. The method of the encoder GPM mode is not limited in this application.

[0236] In the embodiment of the present application, the encoder needs to determine the GPM usage flag or derive whether the GPM is used through other flags. If the current block uses the selected GPM mode, the selected GPM division mode information is encoded, and the MV information used by the two divided parts of the current block or other GPM-related information is encoded, wherein the method of encoding the GPM mode is not limited in this application.

[0237] In the embodiment of the present application, for the curve division mode, the curve division use flag indicates whether the curve division mode is adopted. Here, the curve division use flag can be determined according to the translation amplitude. When there is a translation amplitude, the generated curve division use flag is a used flag. When there is no translation amplitude or the translation amplitude is less than a certain value, the generated curve division use flag is a non-used flag, and the curve division information is not encoded. Among them, the translation amplitude is determined based on the moving distance and the moving vector.

[0238] Exemplarily, if the translation amplitude is not 0, the encoding curve division use flag is 1, and the translation amplitude and the curve shape index are encoded, wherein the encoding order of the translation amplitude and the curve shape index is not limited. If the translation amplitude is not 0, the encoding curve division use flag is 0, and the encoding curve division information is stopped.

[0239] In some embodiments of the present application, the encoder may first encode the translation amplitude. If the translation amplitude is not 0 (such as 1), it indicates that curve division is used and the index of the curve shape is continued to be encoded. If the translation amplitude is 0, the encoding of the curve division information is stopped.

[0240] It should be noted that there may be multiple combinations of the order of the information of the coding curve division and the division mode of the GPM and the information of the MV used by the two division parts of the current block or other information related to the GPM, which is not limited in this application. The coding of the division mode of the GPM and the information of the MV used by the two division parts of the current block or other information related to the GPM is not limited in this application.

[0241] That is, for the current block, how to determine whether to use curve division under the premise of using GPM. One possible implementation is to use a curve division use flag. In this method, the range of the translation amplitude does not include 0. Another possible implementation is to use the translation amplitude. If the translation amplitude is 0, the curve division is not actually used.

[0242] It should be noted that in the embodiment of the present application, multiple curve shapes can be used for curve division, and each curve shape can be represented as an array in the LUT (that is, all curve division information is saved). If it is determined to use curve division, it is necessary to use the curve shape index to determine which curve shape to use. The curve shapes that can be used for curve division may be different depending on the GPM division mode, and the curve shape index is an index in the available shapes. If curve division cannot be used in a certain GPM division mode, then the curve division usage flag, translation amplitude, and curve shape index, etc. are not needed in this GPM mode. If only one curve shape can be used in a certain GPM division mode, then the curve shape index may not be needed in this GPM mode.

[0243] It is understandable that when the encoder adopts the curve division mode to encode the current block, the parameters related to the curve division can be written into the bitstream for use in decoding, thereby improving the decoding efficiency.

[0244] Taking VVC draft 8 as an example, the inter-frame prediction part provided by the embodiment of the present application is described below, and the underlined part below is the embodiment of the main improvement points.

[0245] Merge data syntax

[0246]

[0247]

[0248] Weighted sample prediction process for geometric partitioning mode

[0249] Inputs to this process are:

[0250] –two variables nCbW and nCbH specifying the width and the height of the current coding block,

[0251] –two(nCbW)x(nCbH)arrays predSamplesLA and predSamplesLB,

[0252] –a variable angleIdx specifying the angle index of the geometricpartition,

[0253] –a variable distanceIdx specifying the distance index of thegeometric partition,

[0254] –a variable cIdx specifying colour component index,

[0255] – a variablecurveAmplitudespecifying curveamplitude,

[0256] – a variablecurveIdxspecifyingcurve index .

[0257] Output of this process is the(nCbW)x(nCbH)array pbSamples ofprediction sample values.

[0258] The variables nW,nH,shift1,offset1,hwRatio,displacementX,displacementY,partFlip and shiftHor are derived as follows:

[0259] nW = ( cIdx == 0 ) ? nCbW : nCbW * SubWidthC (1030)

[0260] nH = ( cIdx == 0 ) ? nCbH : nCbH * SubHeightC (1031)

[0261] shift1 = Max(5,17-BitDepth) (1032)

[0262] offset1 = 1<<(shift1-1) (1033)

[0263] hwRatio = nH / nW (1034)

[0264] displacementX = angleIdx (1035)

[0265] displacementY = (angleIdx+8)%32 (1036)

[0266] partFlip = ( angleIdx >= 13 && angleIdx <= 27 ) ? 0 : 1 (1037)

[0267] shiftHor = ( angleIdx%16 == 8 ||(angleIdx%16!=0 && hwRatio > 0 ) ) ?0 : 1 (1038)

[0268] The variables offsetX and offsetY are derived as follows:

[0269] –If shiftHor is equal to 0,the following applies:

[0270] offsetX = (-nW)>>1 (1039)

[0271] offsetY=((-nH)>>1)+

[0272] (angleIdx<16?(distanceIdx*nH)>>3:-((distanceIdx*nH)>>3)) (1040)

[0273] –Otherwise(shiftHor is equal to 1),the following applies:

[0274] offsetX=((-nW)>>1)+

[0275] (angleIdx<16?(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3)) (1041)

[0276] offsetY = (-nH)>>1 (1042)

[0277] The prediction samples pbSamples[x][y]with x=0..nCbW-1and y=0..nCbH-1are derived as follows:

[0278] –The variables xL and yL are derived as follows:

[0279] xL = ( cIdx == 0 ) ? x : x * SubWidthC (1043)

[0280] yL = ( cIdx == 0 ) ? y : y * SubHeightC (1044)

[0281] – The variables offsetCX and offsetCY are derived as follows:

[0282] If((angleIdx>=4 &&angleIdx<12) ||(angleIdx >=20 && angleIdx < 28)) , the following applies:

[0283] offsetCX=0

[0284] offsetCY= curveFactor[curveIdx][xL*N / nW+N / 2nW]* curveAmplitude*nH / N

[0285] – Otherwise (shiftHor is equal to 1), the following applies:

[0286] offsetCX= curveFactor[curveIdx][yL*N / nH+N / 2nH]* curveAmplitude* nW / N

[0287] offsetCY=0

[0288] –The variable wValue specifying the weight of the prediction sampleis derived based on the array disLut specified in Table37 as follows:

[0289] weightIdx=(((xL+offsetX+ offsetCX )<<1)+1)*disLut[displacementX]+

[0290] (((yL+offsetY + offsetCY )<<1)+1))*disLut[displacementY] (1045)

[0291] weightIdxL=partFlip? 32+weightIdx:32-weightIdx (1046)

[0292] wValue=Clip3(0,8,(weightIdxL+4)>>3) (1047)

[0293] –The prediction sample values ​​are derived as follows:

[0294] pbSamples[x][y]=Clip3(0,(1< <BitDepth)-1, (predSamplesLA[x][y]*wValue+ (1048)

[0295] predSamplesLB[x][y]*(8-wValue)+offset1)>>shift1)

[0296] Table37-Specification of the geometric partitioning distance arraydisLut.

[0297] idx 0 2 3 4 5 6 8 10 11 12 13 14 disLut[idx] 8 8 8 4 4 2 0 -2 -4 -4 -8 -8 idx 16 18 19 20 21 22 24 26 27 28 29 30 disLut[idx] -8 -8 -8 -4 -4 -2 0 2 4 4 8 8

[0298] A possible implementation is that N=64.

[0299] like Fig. 9 As shown, the embodiment of the present application provides a decoder 1, comprising:

[0300] The decoding unit 10 is used to parse the bit stream and determine the prediction decoding parameters of the current block;

[0301] The first determination unit 11 is configured to determine, when the prediction decoding parameter indicates that the inter-frame geometric partitioning prediction mode GPM is used to determine the inter-frame prediction of the current block, the GPM mode parameter and the curve partitioning information of the current block; determine the first reference block and the second reference block based on the GPM mode parameter; determine the first prediction parameter corresponding to the first reference block and the second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve partitioning information;

[0302] The first prediction unit 12 is used to determine the prediction value of the current block based on the first prediction parameter and the second prediction parameter.

[0303] In some embodiments of the present application, the first prediction parameter includes: a first weight value or first boundary position information; the second prediction parameter includes: a second weight value or second boundary position information.

[0304] In some embodiments of the present application, the first prediction parameter includes: a first weight value; the second prediction parameter includes: a second weight value;

[0305] The first determination unit 11 is also used to determine a preset weight distribution corresponding to the current GPM mode according to the GPM mode parameters; determine the size information of the current block; update the preset weight distribution according to the size information, the GPM mode parameters and the curve division information to obtain a curve weight distribution; and obtain the first weight value and the second weight value according to the curve weight distribution.

[0306] In some embodiments of the present application, the curve division information includes: a translation amplitude and a curve shape index; the first determination unit 11 is also used to determine an offset direction according to the GPM mode parameters; determine an offset amount according to the size information, the translation amplitude and the curve shape index; according to the offset direction, the preset weight of each pixel position in the preset weight distribution is offset by the offset amount to obtain an updated curve weight of each pixel position, thereby obtaining the curve weight distribution.

[0307] In some embodiments of the present application, the first prediction unit 12 is further used to determine the prediction value of the current block by combining the first weight value, the first pixel value of the first reference block, the second weight value and the second pixel value of the second reference block.

[0308] In some embodiments of the present application, the first prediction parameter includes: first boundary position information; the second prediction parameter includes: second boundary position information;

[0309] The first determination unit 11 is also used to obtain the size information of the current block; determine the division boundary position information based on the GPM mode parameters and the size information; offset the division boundary position information according to the GPM mode parameters, the size information and the curve division information to determine the curve division boundary position information; determine the first boundary position information and the second boundary position information based on the curve division boundary position information.

[0310] In some embodiments of the present application, the first prediction unit 12 is further used to divide the current block according to the first boundary position information and the second boundary position information to obtain a first block and a second block; predict the first block based on the first pixel value of the first reference block, and predict the second block based on the second pixel value of the second reference block, so as to determine the prediction value of the current block.

[0311] In some embodiments of the present application, the curve division information includes: a translation amplitude and a curve shape index; the first determination unit 11 is also used to determine an offset direction according to the GPM mode parameters; determine an offset amount according to the size information, the translation amplitude and the curve shape index; according to the offset direction, the position of each pixel point of the division boundary position information is offset by the offset amount to obtain the curve division boundary position information.

[0312] In some embodiments of the present application, the first determination unit 11 is further used to determine that the offset direction is a horizontal direction when the angle used by the GPM mode parameter to represent GPM meets a horizontal judgment condition; and to determine that the offset direction is a vertical direction when the angle used by the GPM mode parameter to represent GPM meets a vertical judgment condition.

[0313] In some embodiments of the present application, the size information includes the length and width of the current block;

[0314] The first determination unit 11 is further used to determine, when the offset direction is a horizontal direction, a first pixel position located in a preset block at each pixel position in a non-offset direction according to each pixel position, the length and the number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determine first curve information corresponding to the first pixel position according to the curve shape index; determine a first movement ratio based on the width and the number of preset offset points; and determine the offset amount according to the first curve information, the translation amplitude and the first movement ratio, wherein the offset amount does not exceed the width of the current block.

[0315] In some embodiments of the present application, the first determination unit 11 is further used to determine a first scaling ratio based on each pixel position, the length, and the number of preset offset points; determine a first sub-offset based on the length and the number of preset offset points; and determine, based on the first scaling ratio and the first sub-offset, that each pixel position is located in the first pixel position in the preset block in a non-offset direction.

[0316] In some embodiments of the present application, the size information includes the length and width of the current block;

[0317] The first determination unit 11 is further used to determine, when the offset direction is a vertical direction, a second pixel position located in a preset block at each pixel position in a non-offset direction according to each pixel position, the width and the number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determine second curve information corresponding to the second pixel position according to the curve shape index; determine a second movement ratio based on the length and the number of preset offset points; and determine the offset amount according to the second curve information, the translation amplitude and the second movement ratio, wherein the offset amount does not exceed the length of the current block.

[0318] In some embodiments of the present application, the first determination unit 11 is further used to determine a second scaling ratio based on each pixel position, the width, and the number of preset offset points; determine a second sub-offset based on the width and the number of preset offset points; and determine, based on the second scaling ratio and the second sub-offset, the second pixel position of each pixel position in the preset block in a non-offset direction.

[0319] In some embodiments of the present application, the curve division information is a division method in curve division;

[0320] The curve division is to translate the pixel points on the direct division line in the preset straight line geometric division to obtain a continuous curve division line to achieve geometric division.

[0321] In some embodiments of the present application, each pixel point of the straight line dividing line is continuously translated according to different moving distances and the same moving direction to obtain the curved line dividing line; or,

[0322] Each pixel point of the straight line dividing line is continuously translated according to different movement vectors to obtain the curved line dividing line.

[0323] In some embodiments of the present application, the moving distance is the distance between the corresponding pixel position on the preset curve shape and the straight line dividing line;

[0324] The movement vector is a preset vector corresponding to a pixel position on a preset curve shape.

[0325] In some embodiments of the present application, the moving direction is formed by a combination of a horizontal direction and a vertical direction; or,

[0326] The moving direction is related to the division direction of the GPM.

[0327] In some embodiments of the present application, when the moving distance is not an integer pixel, the sub-pixel position is interpolated or rounded; or,

[0328] The moving distance is positively correlated with the length of the current block; or,

[0329] The moving distance is positively correlated with the smallest value between the length and the width of the current block.

[0330] It can be understood that the decoder predicts the current block by using curve division in the GPM mode so that the divided boundary is more closely aligned with the edge of the object or the edge of two parts with different motion vectors, thereby improving the accuracy of encoding and decoding.

[0331] In practical applications, such as Fig.10 As shown, an embodiment of the present application also provides a decoder, including: a first processor 13 and a first memory 14 storing executable instructions of the first processor 13, the first memory 14 relies on the first processor 13 to perform operations through a communication bus 15, and when the executable instructions are executed by the first processor 13, the inter-frame prediction method described on the decoder side is executed.

[0332] Among them, the first processor can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (ASICs), single or multiple general-purpose integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor can execute the corresponding steps of the inter-frame prediction method on the decoder side in the aforementioned embodiment.

[0333] An embodiment of the present application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by one or more first processors, the first processors execute an inter-frame prediction method on the decoder side.

[0334] The components in the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software function modules.

[0335] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage media include: ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM), etc., which can store program codes, and the embodiments of the present disclosure are not limited thereto.

[0336] like Fig.11 As shown, the embodiment of the present application further provides an encoder 2, comprising:

[0337] The second determination unit 20 is configured to determine a prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter-frame prediction of the current block is determined using the inter-frame geometric partitioning prediction mode GPM, determine the GPM mode parameter and curve partitioning information of the current block; determine a first reference block and a second reference block based on the GPM mode parameter; determine a first prediction parameter corresponding to the first reference block and a second prediction parameter corresponding to the second reference block based on the GPM mode parameter and the curve partitioning information;

[0338] The second prediction unit 21 is used to determine the prediction value of the current block based on the first prediction parameter and the second prediction parameter.

[0339] In some embodiments of the present application, the encoder further includes: an encoding unit 22;

[0340] The encoding unit 22 is further used for, after determining the GPM mode parameters, curve division information and motion related information of the current block, when the prediction mode corresponding to the minimum rate distortion cost result is the curve division mode of the GPM prediction mode, generating the curve division usage flag according to the curve division information, and encoding the curve division information; and inputting the curve division usage flag and the encoded curve division information into the bitstream.

[0341] In some embodiments of the present application, the curve division information includes: a translation amplitude and a curve shape index; the encoding unit 22 is also used to generate the curve division usage flag according to the translation amplitude; when the curve division usage flag representation is used, the translation amplitude and the curve shape index are encoded to obtain the encoded curve division information; when the curve division usage flag representation is not used, the encoding of the translation amplitude and the curve shape index is stopped.

[0342] In some embodiments of the present application, the curve division information includes: a translation amplitude and a curve shape index; the encoding unit 22 is further used to encode the translation amplitude to obtain an encoded translation amplitude;

[0343] When the translation amplitude characterization uses curve division, the curve shape index is encoded to obtain the encoded curve shape index, wherein the encoded translation amplitude and the encoded curve shape index are the encoded curve division information; when the translation amplitude characterization uses curve division and the curve division characterization is not used, the encoding of the curve shape index is stopped.

[0344] In some embodiments of the present application, the first prediction parameter includes: a first weight value or first boundary position information;

[0345] The second prediction parameter includes: a second weight value or second boundary position information.

[0346] In some embodiments of the present application, the first prediction parameter includes: a first weight value; the second prediction parameter includes: a second weight value;

[0347] The second determination unit 20 is further used to determine a preset weight distribution corresponding to the current GPM mode according to the GPM mode parameters; determine the size information of the current block; update the preset weight distribution according to the size information, the GPM mode parameters and the curve division information to obtain a curve weight distribution; and obtain the first weight value and the second weight value according to the curve weight distribution.

[0348] In some embodiments of the present application, the curve division information includes: a translation amplitude and a curve shape index; the second determination unit 20 is also used to determine an offset direction according to the GPM mode parameters; determine an offset amount according to the size information, the translation amplitude and the curve shape index; according to the offset direction, the preset weight of each pixel position in the preset weight distribution is offset by the offset amount to obtain an updated curve weight of each pixel position, thereby obtaining the curve weight distribution.

[0349] In some embodiments of the present application, the second prediction unit 21 is further used to determine the prediction value of the current block by combining the first weight value, the first pixel value of the first reference block, the second weight value and the second pixel value of the second reference block.

[0350] In some embodiments of the present application, the first prediction parameter includes: first boundary position information; the second prediction parameter includes: second boundary position information; the second determination unit 20 is also used to obtain the size information of the current block; based on the GPM mode parameter and the size information, determine the division boundary position information; according to the GPM mode parameter, the size information and the curve division information, offset the division boundary position information to determine the curve division boundary position information; based on the curve division boundary position information, determine the first boundary position information and the second boundary position information.

[0351] In some embodiments of the present application, the second prediction unit 21 is further configured to divide the current block according to the first boundary position information and the second boundary position information to obtain a first block and a second block;

[0352] The first block is predicted based on a first pixel value of a first reference block, and the second block is predicted based on a second pixel value of a second reference block, thereby determining a prediction value of the current block.

[0353] In some embodiments of the present application, the curve division information includes: a translation amplitude and a curve shape index; the second determination unit 20 is also used to determine an offset direction according to the GPM mode parameters; determine an offset amount according to the size information, the translation amplitude and the curve shape index; according to the offset direction, the position of each pixel point of the division boundary position information is offset by the offset amount to obtain the curve division boundary position information.

[0354] In some embodiments of the present application, the second determination unit 20 is further used to determine that the offset direction is a horizontal direction when the angle used by the GPM mode parameter to represent GPM meets a horizontal judgment condition; and to determine that the offset direction is a vertical direction when the angle used by the GPM mode parameter to represent GPM meets a vertical judgment condition.

[0355] In some embodiments of the present application, the size information includes the length and width of the current block; the second determination unit 20 is also used to determine, when the offset direction is a horizontal direction, a first pixel position in the preset block at each pixel position in a non-offset direction according to each pixel position, the length and the number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determine, according to the curve shape index, first curve information corresponding to the first pixel position; determine a first movement ratio based on the width and the number of preset offset points; determine the offset amount according to the first curve information, the translation amplitude and the first movement ratio, wherein the offset amount does not exceed the width of the current block.

[0356] In some embodiments of the present application, the second determination unit 20 is further used to determine a first scaling ratio based on each pixel position, the length, and the number of preset offset points; determine a first sub-offset based on the length and the number of preset offset points; and determine, based on the first scaling ratio and the first sub-offset, that each pixel position is located in the first pixel position in the preset block in a non-offset direction.

[0357] In some embodiments of the present application, the size information includes the length and width of the current block;

[0358] The second determination unit 20 is further used to determine, when the offset direction is a vertical direction, a second pixel position located in a preset block at each pixel position in a non-offset direction according to each pixel position, the width and the number of preset offset points, wherein the non-offset direction is perpendicular to the offset direction; determine second curve information corresponding to the second pixel position according to the curve shape index; determine a second movement ratio based on the length and the number of preset offset points; and determine the offset amount according to the second curve information, the translation amplitude and the second movement ratio, wherein the offset amount does not exceed the length of the current block.

[0359] In some embodiments of the present application, the second determination unit 20 is further used to determine a second scaling ratio according to each pixel position, the width, and the number of preset offset points; determine a second sub-offset according to the width and the number of preset offset points; and determine, based on the second scaling ratio and the second sub-offset, the second pixel position of each pixel position in the preset block in the non-offset direction.

[0360] In some embodiments of the present application, the curve division information is a division method in curve division;

[0361] The curve division is to translate the pixel points on the direct division line in the preset straight line geometric division to obtain a continuous curve division line to achieve geometric division.

[0362] In some embodiments of the present application, each pixel point of the straight line dividing line is continuously translated according to different moving distances and the same moving direction to obtain the curved line dividing line; or,

[0363] Each pixel point of the straight line dividing line is continuously translated according to different movement vectors to obtain the curved line dividing line.

[0364] In some embodiments of the present application, the moving distance is the distance between the corresponding pixel position on the preset curve shape and the straight line dividing line;

[0365] The movement vector is a preset vector corresponding to a pixel position on a preset curve shape.

[0366] In some embodiments of the present application, the moving direction is formed by a combination of a horizontal direction and a vertical direction; or,

[0367] The moving direction is related to the division direction of the GPM.

[0368] In some embodiments of the present application, when the moving distance is not an integer pixel, the sub-pixel position is interpolated or rounded; or,

[0369] The moving distance is positively correlated with the length of the current block; or,

[0370] The moving distance is positively correlated with the smallest value between the length and the width of the current block.

[0371] It can be understood that the encoder predicts the current block by using curve division in the GPM mode, so that the divided boundary is more closely aligned with the edge of the object or the edge of two parts with different motion vectors, thereby improving the encoding accuracy.

[0372] In practical applications, such as Fig.12As shown, an embodiment of the present application also provides an encoder, including: a second processor 23 and a second memory 24 storing executable instructions of the second processor 23, the second memory 24 relies on the second processor 23 to perform operations through a communication bus 25, and when the executable instructions are executed by the second processor 23, the inter-frame prediction method on the above-mentioned encoder side is executed.

[0373] Among them, the second processor can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple dedicated integrated circuits, single or multiple general-purpose integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor can execute the corresponding steps of the inter-frame prediction method on the decoder side in the aforementioned embodiment.

[0374] An embodiment of the present application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by one or more second processors, the second processors execute the inter-frame prediction method corresponding to the encoder.

[0375] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0376] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0377] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0378] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0379] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. An inter-frame prediction method, characterized in that: Applicable to decoders, including: Parse the bitstream and determine the prediction mode parameters of the current block; When the prediction mode parameter indicates that the prediction value of the current block is determined using the inter-frame geometric partitioning prediction mode GPM, determining the GPM mode parameter and curve partitioning information of the current block, wherein the GPM mode parameter includes angle index information and motion information of the current block; Determining a first reference block and a second reference block according to the motion information of the current block; Determine, according to the angle index information, the curve division information, and the size information of the current block, a first weight value corresponding to a current point in the current block in the first reference block and a second weight value corresponding to the current point in the second reference block; Determine a prediction value of the current point in the current block according to a first prediction value of the current point in the first reference block, the first weight value, a second prediction value of the current point in the second reference block, and the second weight value.

2. The method according to claim 1, characterized in that The determining, according to the angle index information, the curve division information, and the size information of the current block, a first weight value corresponding to a current point in the current block in the first reference block and a second weight value corresponding to the current point in the second reference block, comprises: Determine the offset direction according to the angle index information; Determining an offset according to the size information of the current block and the curve division information; According to the offset direction, the offset amount and the position information of the current point, a first weight value corresponding to the current point in the first reference block and a second weight value corresponding to the current point in the second reference block are determined.

3. The method according to claim 2, characterized in that The determining the offset direction according to the angle index information includes: When the angle index information indicates that the angle used by the GPM satisfies a horizontal determination condition, determining that the offset direction is a horizontal direction; When the angle index information represents that the angle used by the GPM satisfies the vertical determination condition, the offset direction is determined to be a vertical direction.

4. The method according to claim 1, characterized in that: The motion information of the current block includes first motion information and second motion information. The first reference block is determined according to the first motion information, and the second reference block is determined according to the second motion information. The first motion information is determined according to first index information of the current block, and the second motion information is determined according to second index information of the current block. The first index information and the second index information are obtained by parsing a bit stream.

5. An inter-frame prediction method, characterized in that: Applications for encoders include: Determining prediction mode parameters for the current block; When the prediction mode parameter indicates that the prediction value of the current block is determined using the inter-frame geometric partitioning prediction mode GPM, determining the GPM mode parameter and curve partitioning information of the current block, wherein the GPM mode parameter includes angle index information and motion information of the current block; Determining a first reference block and a second reference block according to the motion information of the current block; Determine, according to the angle index information, the curve division information, and the size information of the current block, a first weight value corresponding to a current point in the current block in the first reference block and a second weight value corresponding to the current point in the second reference block; Determine a prediction value of the current point in the current block according to a first prediction value of the current point in the first reference block, the first weight value, a second prediction value of the current point in the second reference block, and the second weight value.

6. The method according to claim 5, characterized in that The determining, according to the angle index information, the curve division information, and the size information of the current block, a first weight value corresponding to a current point in the current block in the first reference block and a second weight value corresponding to the current point in the second reference block, comprises: Determine the offset direction according to the angle index information; Determining an offset according to the size information of the current block and the curve division information; According to the offset direction, the offset amount and the position information of the current point, a first weight value corresponding to the current point in the first reference block and a second weight value corresponding to the current point in the second reference block are determined.

7. The method according to claim 6, characterized in that The determining the offset direction according to the angle index information includes: When the angle index information indicates that the angle used by the GPM satisfies a horizontal determination condition, determining that the offset direction is a horizontal direction; When the angle index information represents that the angle used by the GPM satisfies the vertical determination condition, the offset direction is determined to be a vertical direction.

8. The method according to claim 5, characterized in that The motion information of the current block includes first motion information and second motion information, the first reference block is determined according to the first motion information, and the second reference block is determined according to the second motion information, The method further includes: encoding first index information corresponding to the first motion information and second index information corresponding to the second motion information, and writing the obtained encoding bits into a bitstream.

9. A decoder, characterized in that: include: A first processor and a first memory storing executable instructions of the first processor, wherein the first memory relies on the first processor to perform operations through a communication bus, and when the executable instructions are executed by the first processor, the inter-frame prediction method described in any one of claims 1 to 4 is executed.

10. An encoder, characterized in that: include: A second processor and a second memory storing executable instructions of the second processor, the second memory relies on the second processor to perform operations through a communication bus, and when the executable instructions are executed by the second processor, the inter-frame prediction method described in any one of claims 5 to 8 is executed.

11. A computer-readable storage medium, wherein: The computer-readable storage medium stores a code stream generated by the encoding method according to any one of claims 5 to 8.

Images