A decoding and encoding method and device thereof

By acquiring and processing motion information in video encoding technology, determining the target motion information for decoding or encoding, the problem of low motion vector prediction quality in inter-frame encoding is solved, and decoding and coding efficiency and performance are improved.

CN114866777BActive Publication Date: 2024-11-22HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210351116.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-09-20
Publication Date
2024-11-22
Estimated Expiration
2038-09-20

AI Technical Summary

Technical Problem

In the existing video encoding technology, the motion vector prediction quality is not high during inter-frame encoding, resulting in prediction errors and affecting decoding efficiency and encoding performance.

Method used

By obtaining the motion model at the decoding and encoding end, establishing a list of motion information, selecting alternative motion information and difference information, determining the original motion information, and determining the target motion information based on the information for decoding or encoding, so as to improve the accuracy of the motion information.

Benefits of technology

Improve the accuracy of motion information, reduce prediction errors, improve decoding and encoding efficiency, and reduce delay.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114866777B_ABST
    Figure CN114866777B_ABST
Patent Text Reader

Abstract

The present application provides a decoding and encoding method and device thereof, the method comprising: obtaining a motion model of a current image block; establishing a motion information list of the current image block according to the motion model; selecting candidate motion information from the motion information list; determining the original motion information of the current image block according to the difference information between the selected candidate motion information and the current image block; determining the target motion information of the current image block according to the original motion information; decoding the encoded bit stream according to the original motion information or the target motion information. Through the technical solution of the present application, the problems of low prediction quality and prediction errors are solved, the decoding efficiency can be improved, the decoding delay can be reduced, and the encoding and decoding performance can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video coding and decoding technology, and in particular to a decoding and encoding method and device thereof. Background Art

[0002] In order to save space, video images are transmitted after being encoded. The complete video encoding method may include prediction, transformation, quantization, entropy coding, filtering and other processes. Among them, predictive coding includes intra-frame coding and inter-frame coding. Inter-frame coding uses the correlation of the video time domain to predict the pixels of the current image using the pixels of the adjacent encoded images to effectively remove the redundancy of the video time domain.

[0003] In inter-frame coding, a motion vector (MV) can be used to represent the relative displacement between the current image block of the current frame video image and the reference image block of the reference frame video image. For example, there is a strong temporal correlation between the video image A of the current frame and the video image B of the reference frame. When the image block A1 (current image block) of the video image A needs to be transmitted, a motion search can be performed in the video image B to find the image block B1 (i.e., the reference image block) that best matches the image block A1, and determine the relative displacement between the image blocks A1 and B1, which is the motion vector of the image block A1.

[0004] The encoder can send the motion vector to the decoder instead of sending the image block A1 to the decoder, and the decoder can obtain the image block A1 according to the motion vector and the image block B1. Obviously, since the number of bits occupied by the motion vector is less than the number of bits occupied by the image block A1, the above method can save a lot of bits.

[0005] However, if the video image A is divided into a large number of image blocks, more bits will be occupied when transmitting the motion vector of each image block. In order to further save bits, the spatial correlation between the candidate image blocks can also be used to predict the motion vector of the image block A1. For example, the motion vector of the image block A2 adjacent to the image block A1 can be determined as the motion vector of the image block A1. Based on this, the encoding end can send the index value of the image block A2 to the decoding end, and the decoding end can determine the motion vector of the image block A2 based on the index value, which is the motion vector of the image block A1. Since the number of bits occupied by the index value of the image block A2 is less than the number of bits occupied by the motion vector, the above method can further save bits.

[0006] Since the motion of image block A1 may be different from that of image block A2, that is, the motion vector of image block A2 may be inconsistent with the motion vector of image block A1, determining the motion vector of image block A2 as the motion vector of image block A1 may result in low prediction quality and prediction errors. Summary of the invention

[0007] The present application provides a decoding and encoding method and device thereof, which solves the problems of low prediction quality and prediction errors, and can improve decoding efficiency, reduce decoding delay, and improve encoding and decoding performance.

[0008] The present application provides a decoding method, which is applied to a decoding end, and the method includes:

[0009] Get the motion model of the current image block;

[0010] Establishing a motion information list of the current image block according to the motion model;

[0011] Selecting candidate motion information from the motion information list; determining original motion information of the current image block according to difference information between the selected candidate motion information and the current image block;

[0012] Determining target motion information of the current image block according to the original motion information;

[0013] The encoded bit stream is decoded according to the original motion information or the target motion information.

[0014] The present application provides an encoding method, which is applied to an encoding end, and the method includes:

[0015] Get the motion model of the current image block;

[0016] Establishing a motion information list of the current image block according to the motion model;

[0017] Selecting candidate motion information from the motion information list; determining original motion information of the current image block according to difference information between the selected candidate motion information and the current image block;

[0018] Determining target motion information of the current image block according to the original motion information;

[0019] The current image block is encoded according to the original motion information or the target motion information.

[0020] The present application provides a decoding end device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned method steps.

[0021] The present application provides an encoding end device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned method steps.

[0022] As can be seen from the above technical solutions, in the embodiments of the present application, the target motion information of the current image block can be determined based on the original motion information, and encoding / decoding can be performed based on the original motion information or the target motion information, rather than directly encoding / decoding based on the original motion information, thereby improving the accuracy of the motion information, solving problems such as low prediction quality and prediction errors, and improving encoding performance / decoding performance. In addition, the above method can also improve encoding efficiency / decoding efficiency and reduce encoding delay / decoding delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0024] Figure 1 is a flowchart of a decoding method in one embodiment of the present application;

[0025] Figure 2A-2C is a flowchart of candidate image blocks in one embodiment of the present application;

[0026] Figure 3A-3E is a schematic diagram of a template in one embodiment of the present application;

[0027] Figure 4 is a flowchart of a decoding method in one embodiment of the present application;

[0028] Figure 5 is a flowchart of an encoding method in one embodiment of the present application;

[0029] Figure 6 is a schematic diagram of a video coding framework in one implementation of the present application;

[0030] Figure 7 is a structural diagram of a decoding device in one embodiment of the present application;

[0031] Figure 8 is a structural diagram of an encoding device in one embodiment of the present application;

[0032] Fig. 9 is a hardware structure diagram of a decoding end device in one embodiment of the present application;

[0033] Fig.10 It is a hardware structure diagram of an encoding end device in one implementation of the present application. DETAILED DESCRIPTION

[0034] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than limiting the present application. The singular forms of "a", "said" and "the" used in the present application and claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.

[0035] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, in addition, the word "if" used may be interpreted as "at..." or "when..." or "in response to determination".

[0036] In the embodiment of the present application, a decoding and encoding method is proposed, which may involve the following concepts:

[0037] Motion Vector (MV): In inter-frame coding, a motion vector is used to represent the relative displacement between the current image block of the current frame video image and the reference image block of the reference frame video image. For example, there is a strong temporal correlation between the video image A of the current frame and the video image B of the reference frame. When transmitting the image block A1 (current image block) of the video image A, a motion search can be performed in the video image B to find the image block B1 (reference image block) that best matches the image block A1, and determine the relative displacement between the image block A1 and the image block B1, which is the motion vector of the image block A1. Each divided image block has a corresponding motion vector transmitted to the decoding end. If the motion vector of each image block is independently encoded and transmitted, especially when it is divided into a large number of small-sized image blocks, a considerable number of bits are consumed. In order to reduce the number of bits used to encode the motion vector, the spatial correlation between adjacent image blocks can be used to predict the motion vector of the current image block to be encoded based on the motion vector of the adjacent encoded image blocks, and then the prediction difference is encoded, which can effectively reduce the number of bits representing the motion vector.

[0038] Motion Information: Since the motion vector represents the position offset between the current image block and a reference image block, in order to accurately obtain the information pointing to the image block, in addition to the motion vector, the index information of the reference frame image is also required to indicate which reference frame image to use. In video coding technology, for the current frame image, a reference frame image list can usually be established, and the reference frame image index information indicates which reference frame image in the reference frame image list is used by the current image block. In addition, many coding technologies also support multiple reference image lists, so an index value can also be used to indicate which reference image list is used. This index value can be called a reference direction. In video coding technology, motion-related information such as motion vectors, reference frame indexes, and reference directions can be collectively referred to as motion information.

[0039] Template: In video coding technology, the encoding process is performed on a block-by-block basis. When encoding the current image block, the reconstruction information of the surrounding encoded image blocks can be used. Template refers to the decoding information of a fixed shape around the current image block (adjacent areas in the time domain or spatial domain). The template is exactly the same at the encoding and decoding ends. Therefore, some operations performed using the template at the encoding end can obtain completely consistent results at the decoding end. In other words, the information derived by the encoding end based on the template can be restored losslessly at the decoding end without transmitting additional information, thereby further reducing the number of transmission bits.

[0040] Rate-Distortion Optimized: There are two major indicators for evaluating coding efficiency: bit rate and PSNR (Peak Signal to Noise Ratio). The smaller the bit stream, the greater the compression rate, and the greater the PSNR, the better the quality of the reconstructed image. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. For example, the cost corresponding to the mode: J(mode) = D + λ*R, where D represents Distortion, which can usually be measured using the SSE indicator. SSE refers to the mean square sum of the differences between the reconstructed image block and the source image; λ is the Lagrange multiplier, and R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, motion information, residuals, etc.

[0041] Intra prediction and inter prediction technology: Intra prediction refers to the use of reconstructed pixel values ​​of spatially adjacent image blocks of the current image block (i.e., in the same frame image as the current image block) for predictive coding, while inter prediction refers to the use of reconstructed pixel values ​​of temporally adjacent image blocks of the current image block (in different frame images as the current image block) for predictive coding.

[0042] CTU (Coding Tree Unit) is the maximum coding unit supported by the encoder and the maximum decoding unit supported by the decoder. Furthermore, a frame of image can be divided into several non-overlapping CTUs, and each CTU can be further divided into smaller blocks based on actual conditions.

[0043] The above decoding method and encoding method are described in detail below in conjunction with several specific embodiments.

[0044] Embodiment 1:

[0045] See also Figure 1 As shown, it is a flowchart of a decoding method, which can be applied to a decoding end and may include:

[0046] Step 101, obtaining a motion model of the current image block. The motion model may include but is not limited to: a 2-parameter motion model (such as a 2-parameter motion vector), a 4-parameter motion model (such as a 4-parameter affine model), a 6-parameter motion model (such as an affine model), and an 8-parameter motion model (such as a projection model).

[0047] Step 102: Create a motion information list of the current image block according to the motion model.

[0048] Step 103: Select candidate motion information from the motion information list.

[0049] Step 104, determining the original motion information of the current image block according to the selected candidate motion information and the difference information of the current image block. Specifically, before step 104, the difference information of the current image block may be obtained first, and then the original motion information of the current image block is determined according to the candidate motion information and the difference information.

[0050] The difference information may be a motion information difference, and there is no restriction on this difference information.

[0051] Step 105: Determine target motion information of the current image block according to the original motion information, wherein the target motion information may be motion information different from the original motion information.

[0052] Step 106: Decode the encoded bit stream according to the original motion information or the target motion information.

[0053] It can be seen from the above technical solution that in the embodiment of the present application, the target motion information of the current image block can be determined based on the original motion information, and decoding can be performed based on the original motion information or the target motion information, rather than directly decoding based on the original motion information, thereby improving the accuracy of the motion information, solving problems such as low prediction quality and prediction errors, and improving decoding performance, improving decoding efficiency, and reducing decoding delay.

[0054] Embodiment 2:

[0055] With respect to step 101, obtaining the motion model of the current image block may include, but is not limited to: obtaining the motion model of the current image block according to the mode information of the current image block. Specifically, the mode information may include the size information of the current image block. Based on this, if the size information of the current image block is smaller than the preset size, it can be determined that the motion model of the current image block is a 2-parameter motion model. If the size information of the current image block is not smaller than the preset size, it can be determined that the motion model of the current image block is a 4-parameter motion model or a 6-parameter motion model. Of course, it can also be a motion model of other parameters; or, if the size information of the current image block is not smaller than the preset size, other methods may be used to determine the motion model of the current image block, for example, the motion model of the current image block may be obtained by encoding the first indication information in the bit stream.

[0056] In another example, obtaining the motion model of the current image block may include, but is not limited to: obtaining first indication information from the coded bit stream, the first indication information being used to indicate the motion model; then, the motion model of the current image block may be obtained based on the first indication information. For example, if the encoding end adopts a 2-parameter motion model, the first indication information added by the encoding end in the coded bit stream is a first identifier; if the encoding end adopts a 4-parameter motion model, the first indication information added by the encoding end in the coded bit stream is a second identifier, and so on. Based on this, after the decoding end obtains the first indication information from the coded bit stream, if the first indication information is the first identifier, it is determined that the motion model is a 2-parameter motion model; if the first indication information is the second identifier, it is determined that the motion model is a 4-parameter motion model, and so on.

[0057] Among them, for the encoding end, in order to determine the motion model of the current image block, the encoding end can obtain the motion model of the current image block according to the mode information of the current image block. For the specific method, please refer to the decoding end, which will not be repeated here. In another example, the encoding end can use each motion model (such as a 2-parameter motion model, a 4-parameter motion model, and a 6-parameter motion model) in turn to pre-encode the bit stream, and then determine the coding performance of each motion model based on the rate-distortion principle, and then determine the motion model with the best coding performance as the motion model of the current image block. Of course, the above methods are only two examples of determining the motion model. The encoding end can also use other methods to determine the motion model, which will not be described in detail here.

[0058] Furthermore, after the encoding end determines the motion model of the current image block, the encoding end may add the first indication information to the encoded bit stream, and the first indication information may be used for the motion model of the current image block.

[0059] Embodiment 3:

[0060] Step 102, establishing a motion information list of the current image block according to the motion model, may include:

[0061] Step 1021: Determine a candidate image block corresponding to the current image block according to the motion model.

[0062] Step 1022: Determine candidate motion information of the current image block according to the motion information of the candidate image block, wherein the motion information of the candidate image block includes a motion vector and a motion direction corresponding to a fixed reference frame.

[0063] Step 1023: Add the candidate motion information of the current image block to the motion information list.

[0064] Among them, multiple candidate image blocks corresponding to the current image block can be determined according to the motion model, and alternative motion information of the current image block can be obtained based on each candidate image block. In this way, multiple alternative motion information can be added to the motion information list, that is, the motion information list can include multiple alternative motion information.

[0065] Embodiment 4: When the motion model is a 2-parameter motion model, for step 1021, the candidate image blocks include one of the following or any combination: image blocks adjacent to the current image block in the current frame where the current image block is located; image blocks that are not adjacent to the current image block in the current frame where the current image block is located; image blocks in adjacent frames of the current frame where the current image block is located, for example, reference image blocks in adjacent frames of the current frame where the current image block is located that have the same position as the current image block, and adjacent image blocks of the reference image block.

[0066] For step 1022, the motion information of the candidate image block can be determined as the alternative motion information of the current image block, such as determining the motion information of the image block adjacent to the current image block in the current frame where the current image block is located (decoded motion information) as the alternative motion information of the current image block; determining the motion information of the image block that is not adjacent to the current image block in the current frame where the current image block is located (decoded motion information) as the alternative motion information of the current image block; determining the motion information of the image block in the adjacent frame of the current frame where the current image block is located (decoded motion information) as the alternative motion information of the current image block.

[0067] In addition, the motion information of the candidate image block can be fine-tuned (such as adding 1 to the x component of the motion vector, subtracting 1 from the x component of the motion vector, adding 1 to the y component of the motion vector, subtracting 1 from the y component of the motion vector, etc., without limitation) to obtain new motion information, and the new motion information can be determined as candidate motion information for the current image block. Default motion information can also be determined as candidate motion information for the current image block, such as default motion information such as zero vector, zero reference frame index, and unidirectional motion direction.

[0068] In one example, the motion information of the candidate image block may be the original motion information of the candidate image block or the target motion information of the candidate image block. For example, if the candidate image block is in the same decoding unit as the current image block, the candidate motion information of the current image block may be determined based on the original motion information of the candidate image block; and / or, if the candidate image block is in the previous decoding unit of the current image block, the candidate motion information of the current image block may be determined based on the original motion information of the candidate image block.

[0069] For another example, when decoding the current image block, the target motion information of the candidate image block can be obtained, and the alternative motion information of the current image block can be determined based on the target motion information of the candidate image block. Specifically, if the target motion information of the candidate image block already exists, the target motion information of the candidate image block can be obtained, and the alternative motion information of the current image block can be determined based on the target motion information of the candidate image block (the method for determining the target motion information of the candidate image block can be the same as the method for determining the target motion information of the current image block, which will not be repeated here); if the target motion information of the candidate image block does not exist, the original motion information of the candidate image block can be obtained, and the alternative motion information of the current image block can be determined based on the original motion information of the candidate image block.

[0070] Wherein, when decoding the current image block, if the candidate image block adopts a motion information adjustment method (i.e., the target motion information is obtained through the original motion information), and the decoding process of the candidate image block has been completed, the target motion information of the candidate image block can be obtained, and therefore, the target motion information of the candidate image block is determined as the candidate motion information of the current image block. If the decoding process of the candidate image block has not been completed (such as when the candidate image block is in the same decoding unit of the current image block and the candidate image block is in the previous decoding unit of the current image block, the decoding process may not be completed), the target motion information of the candidate image block is not obtained, and therefore, the original motion information of the candidate image block is determined as the candidate motion information of the current image block, and the target motion information of the candidate image block is not determined as the candidate motion information of the current image block.

[0071] The above decoding unit may be a maximum allowed decoding block, such as a maximum decoding unit (CTU).

[0072] In one example, when the motion information of the candidate image block is fine-tuned to obtain new motion information, the original motion information of the candidate image block can be adjusted, and the alternative motion information of the current image block can be determined based on the adjusted motion information (such as determining the adjusted motion information as the alternative motion information); and / or, the target motion information of the candidate image block can be adjusted, and the alternative motion information of the current image block can be determined based on the adjusted motion information (such as determining the adjusted motion information as the alternative motion information).

[0073] The fine-tuning of the motion information of the candidate image block may include: scaling the motion information of the candidate image block in a specified direction, performing weighting and other transformation processing, and there is no limitation to this.

[0074] For step 1023, the order of adding the alternative motion information of the current image block corresponding to the motion information of each candidate image block can be determined according to the positional relationship between each candidate image block and the current image block; further, each alternative motion information can be added to the motion information list in turn according to the adding order until the number of alternative motion information in the motion information list reaches a preset number.

[0075] For example, the order of adding the alternative motion information can be as follows: alternative motion information corresponding to the motion information of the image blocks adjacent to the current image block in the current frame where the current image block is located; alternative motion information corresponding to the motion information of the image blocks that are not adjacent to the current image block in the current frame where the current image block is located; alternative motion information corresponding to the motion information of the image blocks in the adjacent frames of the current frame where the current image block is located; motion information obtained after fine-tuning based on the motion information of the candidate image blocks; default motion information.

[0076] Based on the above-mentioned order of adding alternative motion information, each alternative motion information can be added to the motion information list in turn. When adding the alternative motion information to the motion information list, it can be compared with the alternative motion information in the motion information list. If the two are the same, the alternative motion information will not be added to the motion information list. If the two are different, the alternative motion information can be added to the motion information list to prevent the same alternative motion information from appearing in the motion information list as much as possible; and so on, until the number of alternative motion information in the motion information list reaches the preset number N, and no alternative motion information will be added.

[0077] Embodiment 5: The motion model is not a 2-parameter motion model, such as a 4-parameter motion model, a 6-parameter motion model, or an 8-parameter motion model.

[0078] For step 1021, the candidate image blocks may include, but are not limited to: image blocks adjacent to the current image block in the current frame where the current image block is located; wherein the motion model of the candidate image block may be the same as the motion model of the current image block, and the candidate image block includes multiple sub-blocks, and different sub-blocks correspond to the same or different motion information. For example, if the motion model of the current image block is a 4-parameter motion model, an image block of the 4-parameter motion model may be selected from the adjacent image blocks, and the selected image block is used as the candidate image block.

[0079] For step 1022, motion information with a matching number of parameters of the motion model may be selected from the motion information of multiple sub-blocks of the candidate image block, and candidate motion information of the current image block may be determined based on the selected motion information. If the motion model is a 4-parameter motion model, the matching number of parameters of the motion model is 2; if the motion model is a 6-parameter motion model, the matching number of parameters of the motion model is 3; if the motion model is an 8-parameter motion model, the matching number of parameters of the motion model is 4; and so on.

[0080] In an example, if the motion model is a 4-parameter motion model, selecting, from the motion information of multiple sub-blocks of the candidate image block, a number of motion information that matches the parameters of the motion model may include:

[0081] Method 1: Select the motion information of the upper left sub-block (such as the upper left corner sub-block) and the motion information of the upper right sub-block (such as the upper right corner sub-block) of the candidate image block. Figure 2A As shown, image blocks A0, A1, B0, B1 and B2 are candidate image blocks, and the motion information of the upper left subblock and the motion information of the upper right subblock can be selected from the motion information of all subblocks of the candidate image blocks.

[0082] Method 2: For the candidate image block located above the current image block, the motion information of the lower left sub-block (such as the lower left corner sub-block) and the motion information of the lower right sub-block (such as the lower right corner sub-block) of the candidate image block are selected; for the candidate image block not located above the current image block, the motion information of the upper left sub-block (such as the upper left corner sub-block) and the motion information of the upper right sub-block (such as the upper right corner sub-block) of the candidate image block are selected.

[0083] See also Figure 2B As shown, image blocks B0, B1 and B2 are candidate image blocks located above the current image block, and the motion information of the lower left sub-block and the motion information of the lower right sub-block can be selected from the motion information of all sub-blocks of the candidate image blocks. Figure 2B As shown, image block A0 and image block A1 are candidate image blocks that are not located above the current image block, and the motion information of the upper left sub-block and the motion information of the upper right sub-block can be selected from the motion information of all sub-blocks of the candidate image blocks. The above method can be used when the candidate image block and the current image block are not in the same decoding unit (CTU).

[0084] Method three: for the candidate image block located above the current image block, select the motion information of the lower left sub-block (such as the lower left corner sub-block) and the motion information of the lower right sub-block (such as the lower right corner sub-block) of the candidate image block; for the candidate image block not located above the current image block, select the motion information of the upper right sub-block (such as the upper right corner sub-block) and the motion information of the lower right sub-block (such as the lower right corner sub-block) of the candidate image block.

[0085] See also Figure 2C As shown, image blocks B0, B1 and B2 are candidate image blocks located above the current image block, and the motion information of the lower left sub-block and the motion information of the lower right sub-block can be selected from the motion information of all sub-blocks of the candidate image blocks. Figure 2C As shown, image block A0 and image block A1 are candidate image blocks that are not located above the current image block, and the motion information of the upper right sub-block and the motion information of the lower right sub-block can be selected from the motion information of all sub-blocks of the candidate image blocks. The above method can be used when the candidate image block and the current image block are not in the same decoding unit (CTU).

[0086] Of course, the sub-blocks may also be selected in any combination of the above three methods.

[0087] In one example, if the motion model is a 6-parameter motion model, selecting the motion information of the number matching the parameters of the motion model from the motion information of multiple sub-blocks of the candidate image block may also include: selecting the motion information of the upper left sub-block (such as the upper left corner sub-block), the motion information of the upper right sub-block (such as the upper right corner sub-block), and the motion information of the lower left sub-block (such as the lower left corner sub-block) of the candidate image block. In addition, if the motion model is an 8-parameter motion model, selecting the motion information of the number matching the parameters of the motion model from the motion information of multiple sub-blocks of the candidate image block may also include: selecting the motion information of the upper left sub-block (such as the upper left corner sub-block), the motion information of the upper right sub-block (such as the upper right corner sub-block), the motion information of the lower left sub-block (such as the lower left corner sub-block), and the motion information of the lower right sub-block (such as the lower right corner sub-block) of the candidate image block. Of course, the above selection method is only an example and is not limited to this.

[0088] In one example, if the motion model is a 4-parameter motion model, determining the candidate motion information of the current image block according to the selected motion information may include: a 4-parameter affine transformation from a point (x, y) to a point (vx, vy) may be expressed as: v x =a x -b y +e;v y =b x +a y +f, (a, b, e, f) are the affine model parameters. In the 4-parameter motion model in the above embodiment, the 4 parameters are (a, b, e, f). Based on this, the motion information (i.e., motion vectors) of the two sub-blocks of the candidate image block can be used to determine the values ​​of the four parameters (a, b, e, f), and then the four parameter values ​​are used as the parameters of the motion model of the current image block.

[0089] Specifically, based on the two sub-blocks of the candidate image block (taking image block B2 as an example), the four parameters of the current image block can be obtained in the following way: for image block B2, if the motion information of the lower left corner is (x0, y0), and the motion information of the right lower left corner is (x1, y1), assuming that the width of image block B2 is W, the coordinates of the lower left corner of image block B2 are (0, 0), then the coordinates of the lower right corner of image block B2 are (w, 0).

[0090] Therefore, the lower left corner sub-block can move from (0, 0) to (x0, y0), and the lower right corner sub-block can move from (w, 0) to (w+x1, y1), that is, By solving the above equation, we can get the four parameter values ​​(a, b, e, f):

[0091] Embodiment 6: With respect to step 103, selecting candidate motion information from the motion information list may include: selecting candidate motion information from the motion information list according to a motion information index; wherein the motion information index may be obtained from the encoded bit stream, or the motion information index may be a default index (such as 1). For example, if the motion information index is 1, the first candidate motion information is selected from the motion information list.

[0092] In one example, the default index can be directly determined as the motion information index. In another example, the second indication information can be obtained from the coded bit stream, and the second indication information is used to indicate the motion information index. For example, if the encoding end adopts the second alternative motion information in the motion information list, the second indication information added by the encoding end to the coded bit stream is 2, indicating the second alternative motion information in the motion information list, and so on. Based on this, after the decoding end obtains the second indication information from the coded bit stream, if the second indication information is 2, it is determined that the motion information index is 2, indicating the second alternative motion information.

[0093] Among them, for the encoding end, in order to determine the motion information index, the encoding end can directly determine the default index as the motion information index; or the encoding end can use each candidate motion information in the motion information list in turn to pre-encode the bit stream, and then determine the encoding performance of each candidate motion information based on the rate-distortion principle, and then determine the position of the candidate motion information with the best encoding performance in the motion information list as the motion information index. Of course, the above methods are just two examples and are not limited to this.

[0094] Embodiment 7: Before step 104, the method may further include: obtaining difference information of the current image block, specifically, selecting difference information from a difference list according to a difference information index; the difference list is configured with a plurality of difference information; the difference information index is obtained from the coded bit stream, or the difference information index is a default difference index. For example, if the difference information index is 1, the first difference information is selected from the difference list.

[0095] Among them, a difference list can be configured at the encoding end / decoding end based on experience. The difference list is configured with multiple difference information. The difference information can be 0, that is, no difference, or can be non-0, and there is no limitation on this.

[0096] In one example, the default difference index can be directly determined as the difference information index. In another example, the third indication information can be obtained from the encoded bit stream, and the third indication information is used to indicate the difference information index. For example, if the encoding end adopts the first difference information in the difference list, the third indication information added by the encoding end to the encoded bit stream is 1, indicating the first difference information in the difference list, and so on. Based on this, after the decoding end obtains the third indication information from the encoded bit stream, if the third indication information is 1, it is determined that the difference information index is 1, indicating the first difference information in the difference list.

[0097] Among them, for the encoding end, in order to determine the difference information index, the encoding end can directly determine the default difference index as the difference information index; or, the encoding end can use each difference information in the difference list in turn to pre-encode the bit stream, and then determine the encoding performance of each difference information based on the rate-distortion principle, and then determine the position of the difference information with the best encoding performance in the difference list as the difference information index. Of course, the above-mentioned methods for determining the difference information index are just two examples and are not limited to this.

[0098] Embodiment 8: With respect to step 104, determining the original motion information of the current image block according to the selected alternative motion information and the difference information of the current image block may include: determining the original motion information of the current image block according to the alternative motion information and the first weight, the difference information and the second weight. Specifically, the original motion information of the current image block is determined by linear weighting of the alternative motion information and the difference information. For example, the original motion information = a*alternative motion information+b*difference information. a is the first weight, and b is the second weight.

[0099] The values ​​of a and b can be adjusted in real time or can be fixed values. The values ​​of a and b can be configured based on experience or obtained through training. In addition, the values ​​of a and b can be equal or different.

[0100] Embodiment 9: With respect to step 105, determining the target motion information of the current image block based on the original motion information may include but is not limited to: step 1051, obtaining the template of the current image block; step 1052, searching for the target motion information based on the template of the current image block with the original motion information as the center.

[0101] In this embodiment, after obtaining the original motion information, the original motion information may not be used as the final motion information of the current image block, but target motion information different from the original motion information may be obtained based on the original motion information, and the target motion information is closest to the motion information of the current image block, so the target motion information may be used as the final motion information of the current image block. Obviously, compared with the method of "using the original motion information as the final motion information of the current image block", the above method of "using the target motion information as the final motion information of the current image block" can solve the problems of low prediction quality and prediction errors.

[0102] Embodiment 10: With respect to step 1051, obtaining a template of the current image block may include:

[0103] Method 1: Use the original motion information of the current image block to obtain the template of the current image block. Specifically, the original motion information of the current image block (such as decoded motion information) can be used to determine the prediction value of the current image block; according to the prediction value of the current image block, the template of the current image block is obtained, for example, the prediction value of the current image block is determined as the template of the current image block. In the decoding process, each image block is decoded one by one. Therefore, the prediction value of the current image block can be determined using the decoded motion information of the current image block. The prediction value can be the reconstruction information and / or prediction information of the current image block. The reconstruction information can include brightness value, chrominance value, etc. The prediction information can be an intermediate value that can obtain the reconstruction information. If the brightness value can be obtained using the intermediate value A, the intermediate value A is the prediction information, and there is no limitation on this.

[0104] Method 2: obtaining motion information of surrounding image blocks of the current image block (spatial surrounding image blocks of the current image block or temporal surrounding image blocks, without limitation), and using the motion information of surrounding image blocks of the current image block to obtain a template of the current image block. Specifically, using the motion information of the surrounding image blocks (such as decoded motion information), the predicted values ​​of the surrounding image blocks are determined; according to the predicted values ​​of the surrounding image blocks, the template of the current image block is obtained, such as determining the predicted values ​​of the surrounding image blocks as the template of the current image block.

[0105] Among them, when decoding the current image block, since the surrounding image blocks of the current image block have been decoded, that is, the motion information of the surrounding image blocks is known, the motion information of the surrounding image blocks, such as motion vectors and reference frame indexes, etc., can be directly obtained, and the motion information of the surrounding image blocks can be used to determine the prediction value of the surrounding image blocks. The prediction value can be the reconstruction information and / or prediction information of the surrounding image blocks. The reconstruction information can include brightness values, chrominance values, etc. The prediction information can be an intermediate value that can obtain the reconstruction information. For example, if the brightness value can be obtained using the intermediate value A, the intermediate value A is the prediction information, and there is no restriction on this. Then, the prediction value of the surrounding image blocks is determined as the template of the current image block.

[0106] If the surrounding image blocks adopt a motion information adjustment method (ie, the target motion information is obtained through the original motion information), the motion information of the surrounding image blocks is the original motion information of the surrounding image blocks.

[0107] Method 3: obtain motion information of surrounding image blocks of the current image block, and use the motion information of surrounding image blocks of the current image block to obtain the template of the current image block. Specifically, when the motion information includes motion vectors and reference frame indexes of surrounding image blocks, the reference frame image corresponding to the surrounding image block is determined according to the reference frame index; a reference image block corresponding to the surrounding image block is obtained from the reference frame image according to the motion vector, and the template of the current image block is obtained according to the reference image block.

[0108] If the surrounding image blocks adopt a motion information adjustment method (ie, obtain target motion information through original motion information), the motion information of the surrounding image blocks is the original motion information of the surrounding image blocks.

[0109] In an example, the surrounding image blocks may include M first surrounding image blocks and N second surrounding image blocks, where M is a natural number greater than or equal to 1, and N is a natural number greater than or equal to 0, or M is a natural number greater than or equal to 0, and N is a natural number greater than or equal to 1; the first surrounding image blocks are the surrounding image blocks on the upper side of the current image block, and the second surrounding image blocks are the surrounding image blocks on the left side of the current image block.

[0110] Utilizing the motion information of the surrounding image blocks of the current image block to obtain the template of the current image block may include: determining a first template according to the motion vector prediction modes and motion information of M first surrounding image blocks; determining a second template according to the motion vector prediction modes and motion information of N second surrounding image blocks; determining the first template as the template of the current image block; or determining the second template as the template of the current image block; or determining the first template and the second template as the template of the current image block after splicing.

[0111] For example, when M is a natural number greater than or equal to 1 and N is 0, a first template can be determined based on the motion vector prediction mode and motion information of the M first surrounding image blocks, and the first template can be determined as the template of the current image block. When N is a natural number greater than or equal to 1 and M is 0, a second template can be determined based on the motion vector prediction mode and motion information of the N second surrounding image blocks, and the second template can be determined as the template of the current image block. When M is a natural number greater than or equal to 1 and N is a natural number greater than or equal to 1, a first template can be determined based on the motion vector prediction mode and motion information of the M first surrounding image blocks, and a second template can be determined based on the motion vector prediction mode and motion information of the N second surrounding image blocks, and the first template can be determined as the template of the current image block; or the second template can be determined as the template of the current image block; or the first template and the second template can be spliced ​​together to determine them as the template of the current image block.

[0112] Among them, the first surrounding image blocks include adjacent image blocks and / or next-neighboring image blocks on the upper side of the current image block; the prediction mode of the adjacent image blocks is an inter-frame mode or an intra-frame mode; the prediction mode of the next-neighboring image blocks is an inter-frame mode. For example, the first surrounding image blocks may include at least one adjacent image block whose prediction mode is an inter-frame mode, for example, all adjacent image blocks on the upper side of the current image block, or the first adjacent image block on the upper side of the current image block, or any one or more adjacent image blocks on the upper side of the current image block. In addition, if the adjacent image blocks on the upper side of the current image block are all intra-frame modes, the first surrounding image blocks may also include at least one next-neighboring image block whose prediction mode is an inter-frame mode, for example, all next-neighboring image blocks on the upper side of the current image block, or the first next-neighboring image block on the upper side of the current image block, or any one or more next-neighboring image blocks on the upper side of the current image block. In addition, if there is an intra-mode adjacent image block on the upper side of the current image block, the first surrounding image block may also include an intra-mode adjacent image block, for example, the first intra-mode adjacent image block on the upper side of the current image block, all intra-mode adjacent image blocks on the upper side of the current image block, etc. Of course, the above is only an example of the first surrounding image block, and there is no limitation to this.

[0113] Among them, the second surrounding image blocks include adjacent image blocks and / or next adjacent image blocks on the left side of the current image block; the prediction mode of the adjacent image blocks is inter-frame mode or intra-frame mode; the prediction mode of the next adjacent image blocks is inter-frame mode. For example, the second surrounding image blocks may include at least one adjacent image block whose prediction mode is inter-frame mode, for example, all adjacent image blocks on the left side of the current image block, or the first adjacent image block on the left side of the current image block, or any one or more adjacent image blocks on the left side of the current image block. In addition, if the adjacent image blocks on the left side of the current image block are all intra-frame modes, the second surrounding image blocks may also include at least one next adjacent image block whose prediction mode is inter-frame mode, for example, all next adjacent image blocks on the left side of the current image block, or the first next adjacent image block on the left side of the current image block, or any one or more next adjacent image blocks on the left side of the current image block. In addition, if there is an adjacent image block in intra-frame mode on the left side of the current image block, the second surrounding image blocks may also include adjacent image blocks in intra-frame mode, for example, the first adjacent image block in intra-frame mode on the left side of the current image block, all adjacent image blocks in intra-frame mode on the left side of the current image block, etc. Of course, the above is only an example of the second surrounding image block, and there is no limitation to this.

[0114] Among them, the adjacent image blocks of the current image block include but are not limited to: the spatial adjacent image blocks of the current image block (i.e., the adjacent image blocks in the same frame of video image); or, the temporal adjacent image blocks of the current image block (i.e., the adjacent image blocks in different frames of video image). The next-neighboring image blocks of the current image block include but are not limited to: the spatial next-neighboring image blocks of the current image block (i.e., the next-neighboring image blocks in the same frame of video image); or, the temporal next-neighboring image blocks of the current image block (i.e., the next-neighboring image blocks in different frames of video image).

[0115] In one example, when M is greater than 1, the first template may include M sub-templates or P sub-templates, and is spliced ​​from M sub-templates or P sub-templates, where P may be the number of first surrounding image blocks in the inter-frame mode, and P is less than or equal to M. For example, when the M first surrounding image blocks are all surrounding image blocks in the inter-frame mode, the first template includes M sub-templates, and is spliced ​​from M sub-templates. When the M first surrounding image blocks include P surrounding image blocks in the inter-frame mode and MP surrounding image blocks in the intra-frame mode, the first template may include M sub-templates (i.e., one sub-template for each surrounding image block), and is spliced ​​from M sub-templates, or the first template may include P sub-templates (i.e., P sub-templates corresponding to P surrounding image blocks in the inter-frame mode), and is spliced ​​from P sub-templates.

[0116] In addition, when M is equal to 1, the first template may include a first sub-template, and the first sub-template may be determined based on the motion vector prediction mode and motion information of the first surrounding image block on the upper side of the current image block; or, the first sub-template may be determined based on the motion vector prediction mode and motion information of any surrounding image block on the upper side of the current image block. Since the first surrounding image block includes at least one adjacent image block or next adjacent image block whose prediction mode is an inter-frame mode, when M is equal to 1, the first template includes the first sub-template corresponding to the adjacent image block or next adjacent image block of the inter-frame mode.

[0117] The motion information may include a motion vector and a reference frame index of the first surrounding image block, and determining the first template according to the motion vector prediction mode and the motion information of the M first surrounding image blocks may include:

[0118] Case 1: For the ith surrounding image block among the M first surrounding image blocks, when it is determined that the motion vector prediction mode of the ith surrounding image block is the inter-frame mode, the reference frame image corresponding to the ith surrounding image block can be determined according to the reference frame index; further, the reference image block corresponding to the ith surrounding image block can be determined from the reference frame image according to the motion vector of the ith surrounding image block; wherein the relative position offset between the reference image block and the ith surrounding image block can be matched with the motion vector of the ith surrounding image block; further, according to the determined reference image block, an image block with a size of a first horizontal length and a first vertical length can be obtained as the ith sub-template included in the first template.

[0119] Case 2: For the ith surrounding image block among the M first surrounding image blocks, when it is determined that the motion vector prediction mode of the ith surrounding image block is the intra mode, the ith surrounding image block may be filled according to a default value (e.g., a default pixel value, where the default value may be a brightness value pre-configured based on experience). Further, based on the image block filled with the default value, an image block having a size of a first horizontal length and a first vertical length may be obtained as the ith sub-template included in the first template.

[0120] Case 3: For the ith surrounding image block among the M first surrounding image blocks, when it is determined that the motion vector prediction mode of the ith surrounding image block is the intra-frame mode, the reference frame image corresponding to the ith surrounding image block can be determined according to the reference frame index corresponding to the ith surrounding image block; further, the reference image block corresponding to the ith surrounding image block can be determined from the reference frame image according to the motion vector corresponding to the ith surrounding image block, wherein the relative position offset between the reference image block and the ith surrounding image block matches (including being equal or approximately equal to) the motion vector corresponding to the ith surrounding image block; further, according to the determined reference image block, an image block with a size of a first horizontal length and a first vertical length can be obtained as the ith sub-template included in the first template; wherein the reference frame index and motion vector corresponding to the ith surrounding image block are the reference frame index and motion vector of the adjacent image block of the ith surrounding image block.

[0121] Among them, the first horizontal length satisfies a first proportional relationship with the horizontal length of the first surrounding image block (such as 1:1, 1:2, 2:1, etc., without limitation), or satisfies a second proportional relationship with the horizontal length of the current image block (such as 1:1, 1:2, 2:1, etc.), or is equal to a first preset length (configured based on experience). The first vertical length satisfies a third proportional relationship with the vertical length of the first surrounding image block (such as 1:1, 1:2, 2:1, etc.), or satisfies a fourth proportional relationship with the vertical length of the current image block (such as 1:1, 1:2, 2:1, etc.), or is equal to a second preset length (i.e., a length configured based on experience). In addition, the first proportional relationship, the second proportional relationship, the third proportional relationship, and the fourth proportional relationship can be set to be the same or different. The first preset length and the second preset length can be set to be the same or different.

[0122] In one example, when N is greater than 1, the second template may include N sub-templates or R sub-templates, and is spliced ​​from N sub-templates or R sub-templates, where R may be the number of second surrounding image blocks in the inter-frame mode, and R is less than or equal to N. For example, when the N second surrounding image blocks are all surrounding image blocks in the inter-frame mode, the second template includes N sub-templates, and is spliced ​​from N sub-templates. When the N second surrounding image blocks include R surrounding image blocks in the inter-frame mode and NR surrounding image blocks in the intra-frame mode, the second template may include N sub-templates (i.e., one sub-template for each surrounding image block), and is spliced ​​from N sub-templates, or the second template may include R sub-templates (i.e., R sub-templates corresponding to R surrounding image blocks in the inter-frame mode), and is spliced ​​from R sub-templates.

[0123] In addition, when N is equal to 1, the second template may include a second sub-template, and the second sub-template may be determined based on the motion vector prediction mode and motion information of the first surrounding image block on the left side of the current image block; or the second sub-template may be determined based on the motion vector prediction mode and motion information of any surrounding image block on the left side of the current image block. Wherein, since the second surrounding image block includes at least one adjacent image block or next adjacent image block whose prediction mode is the inter-frame mode, therefore, when N is equal to 1, the second template includes the second sub-template corresponding to the adjacent image block or next adjacent image block of the inter-frame mode.

[0124] The motion information may include a motion vector and a reference frame index of the second surrounding image block, and determining the second template according to the motion vector prediction mode and the motion information of the N second surrounding image blocks may include:

[0125] Case 1: For the ith surrounding image block among the N second surrounding image blocks, when it is determined that the motion vector prediction mode of the ith surrounding image block is the inter-frame mode, the reference frame image corresponding to the ith surrounding image block can be determined according to the reference frame index; further, the reference image block corresponding to the ith surrounding image block can be determined from the reference frame image according to the motion vector of the ith surrounding image block; wherein the relative position offset between the reference image block and the ith surrounding image block can be matched with the motion vector of the ith surrounding image block; further, according to the determined reference image block, an image block with a size of a second horizontal length and a second vertical length can be obtained as the ith sub-template included in the second template.

[0126] Case 2: For the ith surrounding image block among the N second surrounding image blocks, when it is determined that the motion vector prediction mode of the ith surrounding image block is the intra mode, the ith surrounding image block can be filled according to a default value (for example, a default pixel value, the default value can be a brightness value pre-configured based on experience). Further, based on the image block filled with the default value, an image block with a size of the second horizontal length and the second vertical length can be obtained as the ith sub-template included in the second template.

[0127] Case 3: For the ith surrounding image block among the N second surrounding image blocks, when it is determined that the motion vector prediction mode of the ith surrounding image block is the intra-frame mode, the reference frame image corresponding to the ith surrounding image block can be determined according to the reference frame index corresponding to the ith surrounding image block; further, the reference image block corresponding to the ith surrounding image block can be determined from the reference frame image according to the motion vector corresponding to the ith surrounding image block, wherein the relative position offset between the reference image block and the ith surrounding image block matches (including being equal or approximately equal to) the motion vector corresponding to the ith surrounding image block; further, according to the determined reference image block, an image block with a size of a second horizontal length and a second vertical length can be obtained as the ith sub-template included in the first template; wherein the reference frame index and motion vector corresponding to the ith surrounding image block are the reference frame index and motion vector of the adjacent image block of the ith surrounding image block.

[0128] Among them, the second horizontal length satisfies the fifth proportional relationship with the horizontal length of the second surrounding image block (such as 1:1, 1:2, 2:1, etc., without limitation), or satisfies the sixth proportional relationship with the horizontal length of the current image block (such as 1:1, 1:2, 2:1, etc.), or is equal to the third preset length (configured according to experience). The second vertical length satisfies the seventh proportional relationship with the vertical length of the second surrounding image block (such as 1:1, 1:2, 2:1, etc.), or satisfies the eighth proportional relationship with the vertical length of the current image block (such as 1:1, 1:2, 2:1, etc.), or is equal to the fourth preset length (i.e., the length configured according to experience). In addition, the fifth proportional relationship, the sixth proportional relationship, the seventh proportional relationship and the eighth proportional relationship can be set to be the same or different. The third preset length and the fourth preset length can be set to be the same or different.

[0129] The following describes the template of the current image block in detail in combination with several specific application scenarios.

[0130] Application scenario 1: See Figure 3A As shown, the surrounding image blocks may include all inter-frame mode adjacent image blocks on the upper side of the current image block and all inter-frame mode adjacent image blocks on the left side of the current image block.

[0131] For the current image block A1, if there are adjacent image blocks in the inter-frame mode on the left, for example, image blocks A3 and A4, then the image blocks A3 and A4 in the inter-frame mode can be determined as the surrounding image blocks of the current image block A1. Similarly, if there are adjacent image blocks in the inter-frame mode on the upper side, for example, image block A2, then the image block A2 in the inter-frame mode can be determined as the surrounding image block of the current image block A1.

[0132] When the surrounding image blocks are image blocks A2, A3 and A4, the template of the current image block A1 is obtained according to the motion information of image block A2, the motion information of image block A3 and the motion information of image block A4. For image block A2, the reference frame image corresponding to image block A2 is determined according to the reference frame index, and image block B2 corresponding to image block A2 is selected from the reference frame image. Image block B2 is moved according to the motion vector of image block A2 to obtain reference image block B2' corresponding to image block A2. Reference image block B3' corresponding to image block A3 and reference image block B4' corresponding to image block A4 can be obtained. See Figure 3B As shown, the template of the current image block A1 is obtained according to the reference image block B2', the reference image block B3' and the reference image block B4'.

[0133] In an example, assuming that the horizontal length of the upper template of the current image block A1 is W, and the vertical length is S, the value of W can be configured based on experience, and the value of S can be configured based on experience, and there is no restriction on the values ​​of W and S. For example, W can be the horizontal length of the current image block A1, the horizontal length of the surrounding image block A2, twice the horizontal length of the current image block A1, etc., and S can be the vertical length of the surrounding image block A2, 1 / 3 of the vertical length of the surrounding image block A2, etc. On this basis, see Figure 3C As shown in FIG. 1 , it is a schematic diagram of a template corresponding to the reference image block B2′. Figure 3C In the example, W is taken as the horizontal length of the surrounding image block A2, that is, W is the horizontal length of the reference image block B2'; S is taken as 1 / 3 of the vertical length of the surrounding image block A2, that is, S is 1 / 3 of the vertical length of the reference image block B2'.

[0134] Assume that the horizontal length of the left template of the current image block A1 is R, and the vertical length is H. The value of R can be configured based on experience, and the value of H can be configured based on experience. There is no restriction on the values ​​of R and H. For example, H can be the vertical length of the current image block A1, the vertical length of the surrounding image block A3, and R can be the horizontal length of the surrounding image block A3, 1 / 3 of the horizontal length of the surrounding image block A3, etc. On this basis, see Figure 3C , which is a schematic diagram of a template corresponding to the reference image block B3 ′, takes H as an example of the longitudinal length of the surrounding image block A3 , and takes R as an example of 1 / 3 of the horizontal length of the surrounding image block A3 .

[0135] Similarly, the template page corresponding to the reference image block B4' can be found in Figure 3C As shown, no further details are given here.

[0136] In an example, assuming that there are M surrounding image blocks of different modes on the upper side of the current image block, then for the i-th surrounding image block, assuming that its horizontal length is w i, then determine the prediction mode of the surrounding image blocks.

[0137] If it is intra mode, no sub-template is generated, or it is filled with default values ​​(such as default pixel values, which may be brightness values ​​pre-configured based on experience) as the i-th sub-template of the upper template.

[0138] If it is an inter-frame mode, the motion information of the i-th surrounding image block (such as a motion vector and a reference frame index, etc.) can be obtained, and a horizontal length w is generated based on the motion vector and the reference frame index. i , a template with a vertical length of S is used as the i-th sub-template of the upper template. Specifically, if the motion vector is MV and the reference frame index is idx, then the horizontal length of w with a relative position offset of MV can be found in the idx-th reference image of the current frame. i , a rectangular block with a longitudinal length of S is used as the i-th sub-template of the upper template.

[0139] Assuming that there are N surrounding image blocks of different modes on the left side of the current image block, for the i-th surrounding image block, assuming that the vertical length is h i , the horizontal length is R, then the prediction mode of the surrounding image blocks is determined.

[0140] If it is intra mode, no sub-template is generated, or it is filled with default values ​​(such as default pixel values, which can be brightness values ​​pre-configured based on experience) as the i-th sub-template of the left template.

[0141] If it is an inter-frame mode, the motion information of the i-th surrounding image block (such as motion vector and reference frame index, etc.) can be obtained, and a horizontal length R and a vertical length h are generated based on the motion vector and the reference frame index. i The template is used as the i-th sub-template of the left template. Specifically, if the motion vector is MV and the reference frame index is idx, then the horizontal length R and the vertical length h of the relative position offset MV can be found in the idx-th reference image of the current frame. i The rectangular block is used as the i-th sub-template of the left template.

[0142] Furthermore, the upper template can be formed by splicing all the upper sub-templates, and the left template can be formed by splicing all the left sub-templates. The upper template and the left template are spliced ​​into the template of the current image block.

[0143] Application scenario 2: See Figure 3D As shown, the surrounding image blocks may include the first inter-frame mode adjacent image block on the upper side of the current image block and the first inter-frame mode adjacent image block on the left side of the current image block.

[0144] For the current image block A1, if the first image block A3 on the left is in inter-frame mode, image block A3 can be determined as the surrounding image block of the current image block A1. If the first image block A2 on the upper side is in inter-frame mode, image block A2 can be determined as the surrounding image block of the current image block A1. When the surrounding image blocks are image blocks A2 and image blocks A3, the template of the current image block A1 is obtained according to the motion information of image blocks A2 and the motion information of image blocks A3. The acquisition method is shown in application scenario 1 and will not be repeated here.

[0145] In an example, assuming that the horizontal length of the upper template of the current image block A1 is W, and the vertical length is S, the value of W can be configured based on experience, and the value of S can be configured based on experience, and there is no restriction on the values ​​of W and S. For example, W can be the horizontal length of the current image block A1, can be the horizontal length of the surrounding image block A2, and S can be the vertical length of the surrounding image block A2, can be 1 / 3 of the vertical length of the surrounding image block A2, etc. Figure 3E FIG. 1 is a schematic diagram of a template corresponding to the reference image block B2 ′.

[0146] Assume that the horizontal length of the left template of the current image block A1 is R, and the vertical length is H. The value of R can be configured based on experience, and the value of H can be configured based on experience. There is no restriction on the values ​​of R and H. For example, H can be the vertical length of the current image block A1, or the vertical length of the surrounding image block A3, etc., and R can be the horizontal length of the surrounding image block A3, or 1 / 3 of the horizontal length of the surrounding image block A3, etc. On this basis, see Figure 3E FIG. 1 is a schematic diagram of a template corresponding to the reference image block B3 ′.

[0147] Of course, the above-mentioned application scenarios 1 and 2 are just two examples, and there is no limitation on this. For example, the surrounding image blocks may include an adjacent image block in the inter-frame mode on the upper side of the current image block, a next-neighboring image block in the inter-frame mode on the upper side of the current image block (that is, when the adjacent image block is in the intra-frame mode, the image block corresponding to the adjacent image block is selected as the next-neighboring image block of the current image block), an adjacent image block in the inter-frame mode on the left side of the current image block, and a next-neighboring image block on the left side of the current image block. For another example, if the first adjacent image block on the upper side of the current image block is in the intra-frame mode, and the image block on the upper side of the first adjacent image block is in the inter-frame mode, then the surrounding image blocks may include the next-neighboring image block in the inter-frame mode on the upper side of the current image block; if the first adjacent image block on the left side of the current image block is in the intra-frame mode, and the image block on the left side of the first adjacent image block is in the inter-frame mode, then the surrounding image blocks may include the next-neighboring image block in the inter-frame mode on the left side of the current image block.

[0148] Embodiment 11: With respect to step 1052, in one implementation, searching for target motion information based on the template of the current image block and centering on the original motion information may include but is not limited to the following steps:

[0149] Step 1052A1, determining the original motion information as the center motion information.

[0150] Step 1052A2, determine edge motion information corresponding to the center motion information.

[0151] The edge motion information may be different from the center motion information.

[0152] In an example, the original motion information may include an original motion vector, the center motion information may include a center motion vector, and the edge motion information may include an edge motion vector.

[0153] Wherein, determining the edge motion vector corresponding to the central motion vector may include: shifting the central motion vector (x, y) in different directions by S, so as to obtain edge motion vectors (xS, y), edge motion vectors (x+S, y), edge motion vectors (x, y+S), and edge motion vectors (x, yS) in different directions. For example, in the horizontal direction, the central motion vector (x, y) may be shifted to the left by S to obtain the edge motion vector (xS, y); in the horizontal direction, the central motion vector (x, y) may be shifted to the right by S to obtain the edge motion vector (x+S, y); in the vertical direction, the central motion vector (x, y) may be shifted upward by S to obtain the edge motion vector (x, y+S); in the vertical direction, the central motion vector (x, y) may be shifted downward by S to obtain the edge motion vector (x, yS).

[0154] The initial value of S can be configured based on experience, such as 2, 4, 8, 16, etc.

[0155] Assuming that the center motion vector is (3, 3) and S is 4, the edge motion vectors are edge motion vector (7, 3), edge motion vector (3, 7), edge motion vector (-1, 3), and edge motion vector (3, -1).

[0156] In the above embodiment, in order to search for edge motion information corresponding to the central motion information, the search is performed with the central motion information as the center and offset upward, downward, left, and right. In another example, instead of performing a directionless search, the search may be performed along the direction of the original motion information.

[0157] Step 1052A3, obtain the prediction performance of the center motion information (center motion vector) according to the template of the current image block, and obtain the prediction performance of the edge motion information (edge ​​motion vector) according to the template of the current image block.

[0158] Case 1, obtaining the prediction performance of the center motion vector according to the template of the current image block may include but is not limited to: determining the prediction performance of the center motion vector according to parameter information of the template of the current image block and parameter information of the first target reference block, the first target reference block may be an image block obtained after the reference image block corresponding to the template is offset based on the center motion vector. Specifically, the prediction performance of the center motion vector may be determined according to the parameter information of the template and the parameter information of the first target reference block.

[0159] The above parameter information may be a brightness value; or may be a brightness value and a chromaticity value.

[0160] Assuming that the parameter information is a brightness value, in order to determine the prediction performance of the center motion vector, the brightness value of the template of the current image block and the brightness value of the first target reference block can be obtained first. For example, after obtaining the template of the current image block, the brightness value of each pixel of the template can be obtained, and the reference image block of the template can be obtained. Assuming that the center motion vector is (3, 3), the reference image block can be moved using the center motion vector (3, 3) to obtain an image block X corresponding to the reference image block (such as moving the reference image block to the right by 3 pixels and upward by 3 pixels, and the processed image block is recorded as image block X), and image block X is the first target reference block, and the brightness value of each pixel of image block X can be obtained.

[0161] Based on the brightness value of each pixel of the template and the brightness value of each pixel of the image block X, the prediction performance of the central motion vector can be determined by the following formula: SAD is the sum of available absolute differences and is used to indicate the prediction performance of the center motion vector. i Indicates the brightness value of the i-th pixel of the template, TMP i represents the brightness value of the i-th pixel of the image block X, and M represents the total number of pixels.

[0162] Assuming that the parameter information is brightness value and chromaticity value, the following formula is used The brightness value prediction performance SAD of the center motion vector is determined, and the chrominance value prediction performance of the center motion vector is determined by the following formula: The average of the luminance value prediction performance SAD and the chrominance value prediction performance CSAD is the prediction performance of the center motion vector. Among them, CSAD is the sum of available absolute differences, which is used to represent the chrominance value prediction performance of the center motion vector, CTM iIndicates the chromaticity value of the i-th pixel of the template, CTMP i Represents the chromaticity value of the i-th pixel in the image block X, M c Indicates the total number of pixels.

[0163] Case 2, obtaining the prediction performance of the edge motion vector according to the template of the current image block, may include but is not limited to: determining the prediction performance of the edge motion vector according to the parameter information of the template of the current image block and the parameter information of the second target reference block, the second target reference block may be an image block obtained after the reference image block corresponding to the template is offset based on the edge motion vector. Specifically, the prediction performance of the edge motion vector may be determined according to the parameter information of the template and the parameter information of the second target reference block.

[0164] The above parameter information may be a brightness value; or may be a brightness value and a chromaticity value.

[0165] Case 2 is similar to Case 1, except that: in Case 1, the reference image block of the edge motion vector moving template is used to obtain the second target reference block, and the second target reference block is used to obtain the prediction performance of the edge motion vector, while in Case 2, the reference image block of the center motion vector moving template is used to obtain the first target reference block, and the prediction performance of the center motion vector is obtained using the first target reference block.

[0166] Step 1052A4, determining target motion information (such as target motion vector) from the center motion information and the edge motion information according to the prediction performance of the center motion information and the prediction performance of the edge motion information.

[0167] Specifically, a motion vector with the best prediction performance can be selected from the center motion vector and the edge motion vector; when the motion vector with the best prediction performance is not the original motion vector, the motion vector with the best prediction performance can be determined as the target motion vector; when the motion vector with the best prediction performance is the original motion vector, a motion vector with the second best prediction performance can be selected from the center motion vector and the edge motion vector, and the motion vector with the second best prediction performance can be determined as the target motion vector.

[0168] Embodiment 12: With respect to step 1052, in another implementation, searching for target motion information based on the template of the current image block and centering on the original motion information may include but is not limited to the following steps:

[0169] Step 1052B1, determining the original motion information as the center motion information.

[0170] Step 1052B2, determine edge motion information corresponding to the center motion information.

[0171] Step 1052B3, obtaining prediction performance of center motion information according to the template of the current image block, and obtaining prediction performance of edge motion information according to the template of the current image block.

[0172] Step 1052B1-Step 1052B3 refer to Step 1052A1-Step 1052A3 and will not be repeated here.

[0173] Step 1052B4, determine whether the iteration end condition of the target motion information is met.

[0174] If yes, step 1052B6 may be executed; if no, step 1052B5 may be executed.

[0175] The iteration end condition may include but is not limited to: the number of iterations reaches a number threshold, or the execution time reaches a time threshold, or the parameter S has been modified to a preset value, such as 1.

[0176] Of course, the above are just a few examples of iteration end conditions, and there is no limitation on the iteration end conditions.

[0177] Step 1052B5, select the motion information with the best prediction performance from the center motion information and the edge motion information, determine the motion information with the best prediction performance as the center motion information, and return to step 1052B2.

[0178] When step 1052B2 is executed for the first time, the value of parameter S may be an initial value, such as 16. When step 1052B2 is executed again, the value of parameter S is adjusted, such as adjusting to the last parameter S minus 2, or adjusting to half of the last parameter S. There is no restriction on this, as long as it is less than the last parameter S. The following takes adjusting to half of the last parameter S as an example. Therefore, when step 1052B2 is executed for the second time, the value of parameter S is 8; when step 1052B2 is executed for the third time, the value of parameter S is 4; and so on.

[0179] After adjusting the value of parameter S, it is determined whether the adjusted parameter S is less than or equal to a preset value, such as 1. If not, step 1052B2 is executed based on the adjusted parameter S, and this process is not described in detail. If yes, the value of parameter S is set to 1, and step 1052B2 is executed based on the parameter S (i.e., the value 1), and when executing to step 1052B4, the judgment result is that the iteration end condition is met.

[0180] Step 1052B6, determining target motion information (such as target motion vector) from the center motion information and the edge motion information according to the prediction performance of the center motion information and the prediction performance of the edge motion information.

[0181] Among them, the processing of step 1052B6 can refer to step 1052A4, and will not be repeated here.

[0182] Embodiment 13: With respect to step 105, the current image block is a bidirectional inter-frame prediction block, and the original motion information includes a first reference frame and a second reference frame corresponding to the current image block, and a first motion vector corresponding to the first reference frame and a second motion vector corresponding to the second reference frame; see Figure 4 As shown, it is a flowchart of a decoding method, which may include: determining target motion information of a current image block according to original motion information;

[0183] Step 401, obtaining a first initial reference block (an image block in the first reference frame) corresponding to a current image block from a first reference frame according to a first motion vector, and obtaining a second initial reference block (an image block in the second reference frame) corresponding to the current image block from a second reference frame according to a second motion vector.

[0184] Step 402: Based on the similarity between the first initial reference block and the second initial reference block, in the first reference frame, with the first motion vector as the center, a first target reference block (an image block in the first reference frame) is searched according to a plurality of offset directions and offsets; in the second reference frame, with the second motion vector as the center, a second target reference block (an image block in the second reference frame) is searched according to a plurality of offset directions and offsets. The similarity between the first target reference block and the second target reference block meets a preset requirement.

[0185] In one example, the similarity between the first initial reference block and the second initial reference block can be determined in the following manner: using bilinear interpolation to determine the prediction value of the first initial reference block and the prediction value of the second initial reference block; obtaining the downsampling processing results of the prediction value of the first initial reference block and the prediction value of the second initial reference block, and based on the downsampling processing results, determining the SAD (sum of abstract difference) or MRSAD (mean removed sum of abstract difference) of the first initial reference block and the second initial reference block, and determining the similarity between the first initial reference block and the second initial reference block based on the SAD or MRSAD, that is, the SAD or MRSAD can be the similarity.

[0186] In order to search for the first target reference block and the second target reference block, the following method may be used:

[0187] The first motion vector is determined as the first center motion vector, and the second motion vector is determined as the second center motion vector; then, a first edge motion vector corresponding to the first center motion vector is determined, and a second edge motion vector corresponding to the second center motion vector is determined. Then, a first initial reference block corresponding to the current image block is determined from the first reference frame according to the first center motion vector, and a first prediction value of the first initial reference block is determined. A second initial reference block corresponding to the current image block is determined from the second reference frame according to the second center motion vector, and a second prediction value of the second initial reference block is determined. A third initial reference block corresponding to the current image block is determined from the first reference frame according to the first edge motion vector, and a third prediction value of the third initial reference block is determined. A fourth initial reference block corresponding to the current image block is determined from the second reference frame according to the second edge motion vector, and a fourth prediction value of the fourth initial reference block is determined.

[0188] Furthermore, the first similarity can be determined according to the first prediction value and the second prediction value, and the second similarity can be determined according to the third prediction value and the fourth prediction value. Then, the first target reference block and the second target reference block can be determined according to the first similarity and the second similarity. Specifically, if the first similarity is higher than the second similarity, the first initial reference block is determined as the first target reference block, and the second initial reference block is determined as the second target reference block; if the second similarity is higher than the first similarity, the third initial reference block can be determined as the first target reference block, and the fourth initial reference block can be determined as the second target reference block.

[0189] In one example, before determining the first target reference block and the second target reference block according to the first similarity and the second similarity, it can also be determined whether the termination condition is met (such as the number of iterations reaches a number threshold, or the execution time reaches a time threshold, etc.); if so, the steps of determining the first target reference block and the second target reference block according to the first similarity and the second similarity can be executed; if not, if the first similarity is higher than the second similarity, the steps of determining the first edge motion vector corresponding to the first center motion vector and the second edge motion vector corresponding to the second center motion vector can be executed; if the second similarity is higher than the first similarity, the first edge motion vector can be determined as the first center motion vector, the second edge motion vector can be determined as the second center motion vector, and the steps of determining the first edge motion vector corresponding to the first center motion vector and the second edge motion vector corresponding to the second center motion vector can be executed.

[0190] In the above embodiment, the first similarity may include but is not limited to SAD or MRSAD, and the second similarity may include but is not limited to SAD or MRSAD.

[0191] In the above embodiment, the first similarity is determined according to the first prediction value and the second prediction value, including: the first prediction value and the second prediction value can be downsampled, and the first similarity is determined using the downsampled first prediction value and the downsampled second prediction value; the second similarity is determined according to the third prediction value and the fourth prediction value, including: the third prediction value and the fourth prediction value can be downsampled, and the second similarity is determined using the downsampled third prediction value and the downsampled fourth prediction value.

[0192] In the above embodiments, an interpolation strategy in which the number of horizontal and vertical taps are both less than or equal to N (N is preferably 2 (bilinear interpolation), of course, N can also be other values, without limitation) can be adopted to determine the corresponding first prediction value of the first initial reference block; an interpolation strategy in which the number of horizontal and vertical taps are both less than or equal to N can be adopted to determine the corresponding second prediction value of the second initial reference block; an interpolation strategy in which the number of horizontal and vertical taps are both less than or equal to N can be adopted to determine the corresponding third prediction value of the third initial reference block; and an interpolation strategy in which the number of horizontal and vertical taps are both less than or equal to N can be adopted to determine the corresponding fourth prediction value of the fourth initial reference block.

[0193] In the above embodiment, determining the first edge motion vector corresponding to the first center motion vector and the second edge motion vector corresponding to the second center motion vector may include: Mode 1: performing offset processing (including direction and offset) on the first center motion vector to obtain the first edge motion vector; performing offset processing on the second center motion vector to obtain the second edge motion vector. Mode 2: performing offset processing on the first center motion vector to obtain the first edge motion vector; deriving the difference between the second edge motion vector and the second center motion vector according to the difference between the first edge motion vector and the first center motion vector, and determining the second edge motion vector using the difference and the second center motion vector; for example, if the first edge motion vector corresponds to the difference A with the first center motion vector, then the second edge motion vector and the second center motion vector are the difference A, and therefore, the second edge motion vector is the sum of the second center motion vector and the difference A; for another example, if the first edge motion vector corresponds to the difference A with the first center motion vector, then the second edge motion vector and the second center motion vector are -1*difference A, and therefore, the second edge motion vector is the difference between the second center motion vector and the difference A.

[0194] Step 403: determine the motion information of the first target reference block as the target motion information of the current image block, and determine the motion information of the second target reference block as the target motion information of the current image block.

[0195] Embodiment 14: The current image block may have original motion information M, and the original motion information M may include a first motion vector M1 and a second motion vector M2, wherein the first motion vector M1 may correspond to a reference frame A1, and the second motion vector M2 may correspond to a reference frame A2.

[0196] The first motion vector M1 is determined as the first center motion vector T1, and the second motion vector M2 is determined as the second center motion vector T2. The first center motion vector T1 is offset by S in a certain direction to obtain the first edge motion vector W1, and the second center motion vector T2 is offset by S in a certain direction to obtain the second edge motion vector W2; of course, the above is only an example of determining edge motion information, and there is no limitation to this.

[0197] The first initial reference block corresponding to the current image block can be determined from the reference frame A1 according to the first central motion vector T1, and the first prediction value of the first initial reference block can be determined. The first prediction value of the first initial reference block can be determined by using an interpolation strategy with both horizontal and vertical tap numbers less than or equal to N, that is, the first prediction value (such as pixel value, which can be a brightness value, a color value, etc.) of each pixel in the first initial reference block; specifically, if the search is a pixel search, that is, the minimum search precision is less than 1 pixel, then the pixel in the first initial reference block may not be an integer pixel, such as 0.75, etc. Therefore, in order to know the pixel value of this pixel, an interpolation strategy with both horizontal and vertical tap numbers less than N can be used. For example, if both the horizontal and vertical tap numbers are 2, the above interpolation strategy is a bilinear interpolation strategy, that is, the bilinear interpolation strategy can be used to determine the first prediction value (such as pixel value) of the pixel 0.75. For example, the four real pixel values ​​around the pixel 0.75 can be used to determine the first prediction value.

[0198] Among them, the second initial reference block corresponding to the current image block can be determined from the reference frame A2 according to the second center motion vector T2, and the second prediction value of the second initial reference block can be determined. The third initial reference block corresponding to the current image block can be determined from the reference frame A1 according to the first edge motion vector W1, and the third prediction value of the third initial reference block can be determined. The fourth initial reference block corresponding to the current image block can be determined from the reference frame A2 according to the second edge motion vector W2, and the fourth prediction value of the fourth initial reference block can be determined.

[0199] Further, similarity 1 may be determined according to the first prediction value and the second prediction value, and similarity 2 may be determined according to the third prediction value and the fourth prediction value. When determining similarity 1 and similarity 2, if similarity 1 and similarity 2 are SAD, for each pixel point (such as P1-P100) in the first intermediate image block and each pixel point (such as Q1-Q100) in the second intermediate image block, similarity 1 is determined in the following manner:

[0200] First, the prediction values ​​of pixel points P1-P100 can be downsampled, and the prediction values ​​of pixel points Q1-Q100 can be downsampled. For example, if the downsampling rate is 1, the downsampling results are the prediction values ​​of pixel points P1-P100 and the prediction values ​​of pixel points Q1-Q100; if the downsampling rate is 2, the downsampling results are the prediction values ​​of pixel points P1-P50 and the prediction values ​​of pixel points Q1-Q50; if the downsampling rate is 4, the downsampling results are the prediction values ​​of pixel points P1-P25 and the prediction values ​​of pixel points Q1-Q25; and so on.

[0201] Assuming that the downsampling result is the predicted value of pixel points P1-P25 and the predicted value of pixel points Q1-Q25, the absolute value of the difference between the predicted value of P1 and the predicted value of Q1 can be calculated, such as X1, and the absolute value of the difference between the predicted value of P2 and the predicted value of Q2 can be calculated, such as X2. By analogy, the absolute value of the difference between the predicted value of P25 and the predicted value of Q25 can be calculated, such as X25. Then, the average value of X1, X2...X25 is taken. This average value is SAD, which is similarity 1. Similarly, similarity 2 can be determined in the above manner.

[0202] When determining similarity 1 and similarity 2, if similarity 1 and similarity 2 are MRSAD, for each pixel point in the first intermediate image block (such as P1-P100) and each pixel point in the second intermediate image block (such as Q1-Q100), similarity 1 is determined in the following manner: first, the predicted values ​​of the pixels P1-P100 are downsampled, and the predicted values ​​of the pixels Q1-Q100 are downsampled. For example, if the downsampling rate is 4, the downsampling results are the predicted values ​​of the pixels P1-P25 and the predicted values ​​of the pixels Q1-Q25.

[0203] Determine the mean A of the predicted values ​​of P1-P25, and determine the mean B of the predicted values ​​of Q1-Q25; calculate the difference C1 between the predicted value of P1 and the mean A, the difference D1 between the predicted value of Q1 and the mean B, and calculate the absolute value of the difference between the difference C1 and the difference D2, such as Y1, and so on, calculate the difference C25 between the predicted value of P25 and the mean A, the difference D25 between the predicted value of Q25 and the mean B, and calculate the absolute value of the difference between the difference C25 and the difference D25, such as Y25, and then take the average value of Y1, Y2...Y25, and this average value is MRSAD, that is, similarity 1. Similarly, the above method can be used to determine similarity 2.

[0204] After determining similarity 1 and similarity 2, determine whether the end condition is met; if not, if similarity 1 is higher than similarity 2 (the smaller the similarity value, the higher the similarity), then re-search the first edge motion vector W3 (different from the above-mentioned first edge motion vector W1) based on the first center motion vector T1, and re-search the second edge motion vector W4 (different from the above-mentioned second edge motion vector W2) based on the first center motion vector T2, and re-execute the above steps based on the first center motion vector T1, the first edge motion vector W3, the first center motion vector T2, and the second edge motion vector W4, and will not be repeated here. If similarity 2 is higher than similarity 1, the first edge motion vector W1 is determined as the first center motion vector T3, and the second edge motion vector W2 is determined as the second center motion vector T4. The first edge motion vector W5 is re-searched according to the first center motion vector T3, and the second edge motion vector W6 is re-searched according to the second center motion vector T4. The above steps are re-executed based on the first center motion vector T3, the first edge motion vector W5, the first center motion vector T4, and the second edge motion vector W6, and will not be repeated here.

[0205] After determining whether the end condition is met, if yes, the first target reference block and the second target reference block are determined according to the first similarity and the second similarity (such as similarity 1 and similarity 2). If similarity 1 is higher than similarity 2, the first initial reference block can be determined as the first target reference block, and the second initial reference block can be determined as the second target reference block; if similarity 2 is higher than similarity 1, the third initial reference block can be determined as the first target reference block, and the fourth initial reference block can be determined as the second target reference block.

[0206] Embodiment 15: With respect to step 105, the original motion information includes a first original motion vector and a second original motion vector corresponding to the current image block; the determination of the target motion information of the current image block based on the original motion information may include but is not limited to: determining the first target motion vector of the current image block based on the first original motion vector, and the specific determination method is shown in the above embodiment; then, determining the difference between the first original motion vector and the first target motion vector; determining the second target motion vector based on the second original motion vector and the difference; and determining the target motion information based on the first target motion vector and the second target motion vector.

[0207] For example, the reference direction of the original motion information may be bidirectional, that is, the original motion information may include a first original motion vector and a second original motion vector. Assuming that the first original motion vector is (V0x, V0y) and the second original motion vector is (V1x, V1y), for a certain motion information component of the original motion information, taking the first original motion vector (V0x, V0y) as an example, the implementation methods of Embodiments 9 to 14 may be used to determine the first target motion vector corresponding to the first original motion vector, and this determination process is not limited, and assuming that the first target motion vector is (V0x+offset_x, V0y+offset_y).

[0208] On this basis, it can be determined that the difference between the first target motion vector and the first original motion vector is (offset_x, offset_y). Then, the second target motion vector corresponding to the second original motion vector (V1x, V1y) can be directly derived through (offset_x, offset_y). For example, the second target motion vector can be (V1x-offset_x, V1y-offset_y). Of course, the above is just an example and there is no limitation to this.

[0209] Further, the first target motion vector (V0x+offset_x, V0y+offset_y) and the second target motion vector (V1x-offset_x, V1y-offset_y) may be determined as target motion information.

[0210] Embodiment 16: With respect to step 106, decoding the coded bit stream according to the original motion information or the target motion information may include: the decoding end may obtain fourth indication information from the coded bit stream, the fourth indication information being used to indicate that the coded bit stream is decoded using the original motion information, or that the coded bit stream is decoded using the target motion information. Of course, the above manner is only an example and is not limited thereto.

[0211] For example, if the encoding end uses the original motion information to encode the bit stream, the fourth indication information added by the encoding end to the encoded bit stream may be a first identifier, indicating that the bit stream is encoded using the original motion information; if the encoding end uses the target motion information to encode the bit stream, the fourth indication information added by the encoding end to the encoded bit stream may be a second identifier, indicating that the bit stream is encoded using the target motion information. Based on this, after the decoding end obtains the fourth indication information from the encoded bit stream, if the fourth indication information is the first identifier, the encoded bit stream can be decoded according to the original motion information; if the fourth indication information is the second identifier, the encoded bit stream can be decoded according to the target motion information.

[0212] Among them, for the encoding end, the original motion information can be used to pre-encode the bit stream, and the target motion information can be used to pre-encode the bit stream. In this way, the encoding performance of the original motion information and the encoding performance of the target motion information can be determined based on the rate-distortion principle. If the encoding performance of the original motion information is better than the encoding performance of the target motion information, it is determined that the original motion information is used to encode the bit stream; if the encoding performance of the target motion information is better than the encoding performance of the original motion information, it is determined that the target motion information is used to encode the bit stream. Of course, the above method is only an example and is not limited to this.

[0213] Embodiment 17: After step 106, the final motion information of the current image block may also be determined. For example, if the coded bit stream is decoded according to the original motion information, the original motion information is stored as the final motion information; or, if the coded bit stream is decoded according to the target motion information, the target motion information is stored as the final motion information; wherein the final motion information is used as a decoding reference for subsequent image blocks, that is, the final motion information may be referenced during the processing of other image blocks.

[0214] Embodiment 18:

[0215] See also Figure 5 As shown, it is a schematic diagram of the process of the encoding method, which can be applied to the encoding end and may include:

[0216] Step 501, obtain a motion model of the current image block. The motion model may include but is not limited to: a 2-parameter motion model (such as a 2-parameter motion vector), a 4-parameter motion model (such as a 4-parameter affine model), a 6-parameter motion model (such as an affine model), and an 8-parameter motion model (such as a projection model).

[0217] Step 502: Create a motion information list of the current image block according to the motion model.

[0218] Step 503: Select candidate motion information from the motion information list.

[0219] Step 504, determining the original motion information of the current image block according to the selected candidate motion information and the difference information of the current image block. Specifically, before step 104, the difference information of the current image block may be obtained first, and then the original motion information of the current image block is determined according to the candidate motion information and the difference information.

[0220] Step 505: Determine target motion information of the current image block according to the original motion information, wherein the target motion information may be motion information different from the original motion information.

[0221] Step 506: Encode the current image block according to the original motion information or the target motion information.

[0222] It can be seen from the above technical solution that in the embodiment of the present application, the target motion information of the current image block can be determined based on the original motion information, and encoding can be performed based on the original motion information or the target motion information, rather than directly encoding based on the original motion information, thereby improving the accuracy of the motion information, solving problems such as low prediction quality and prediction errors, and improving encoding performance, improving encoding efficiency, and reducing encoding delay.

[0223] Embodiment 19:

[0224] For step 501, the motion model of the current image block is obtained, and its implementation method can refer to embodiment 2, which will not be described in detail here. For step 502, a motion information list of the current image block is established according to the motion model, and its implementation method can refer to embodiment 3 to embodiment 5, which will not be described in detail here. For step 503, candidate motion information is selected from the motion information list, and its implementation method can refer to embodiment 6, which will not be described in detail here.

[0225] Before step 504, the difference information of the current image block can be obtained. The specific implementation method can be referred to in Example 7, which will not be described in detail here. For step 504, the original motion information of the current image block is determined according to the selected candidate motion information and the difference information. The implementation method can be referred to in Example 8, which will not be described in detail here.

[0226] For step 505, the target motion information of the current image block is determined according to the original motion information. The implementation method thereof can refer to embodiments 9-15, which will not be described in detail here. For step 506, the current image block is encoded according to the original motion information or the target motion information. The implementation method thereof can refer to embodiment 16.

[0227] After encoding the current image block according to the original motion information or the target motion information, the final motion information of the current image block may also be determined. For example, if the current image block is encoded according to the original motion information, the original motion information is stored as the final motion information; or, if the current image block is encoded according to the target motion information, the target motion information is stored as the final motion information.

[0228] The final motion information is used as a coding reference for subsequent image blocks, that is, the final motion information can be referred to during the processing of other image blocks, and there is no restriction on this.

[0229] Embodiment 20:

[0230] See also Figure 6 As shown in FIG. 1 , a video encoding framework can be used to implement the above encoding method. In addition, a video decoding framework and Figure 6Similarly, and no further description is given here, the above decoding method can be implemented using a video decoding framework. Specifically, in the video encoding framework / video decoding framework, modules such as intra-frame prediction, motion estimation / motion compensation, reference image buffer, in-loop filtering, reconstruction, transformation, quantization, inverse transformation, back-quantization, and entropy encoder can be included. At the encoding end, the above encoding method can be implemented through the cooperation of these modules; at the decoding end, the above decoding method can be implemented through the cooperation of these modules.

[0231] Embodiment 21:

[0232] Based on the same application concept as the above method, the embodiment of the present application also proposes a decoding device, which is applied to a decoding end, such as Figure 7 As shown, it is a structural diagram of the device, and the device includes:

[0233] An acquisition module 71, used to acquire a motion model of a current image block;

[0234] An establishing module 72, configured to establish a motion information list of the current image block according to the motion model;

[0235] A selection module 73, configured to select candidate motion information from the motion information list;

[0236] A determination module 74, configured to determine original motion information of the current image block according to the selected candidate motion information and difference information of the current image block; and

[0237] Determining target motion information of the current image block according to the original motion information;

[0238] The decoding module 75 is configured to decode the encoded bit stream according to the original motion information or the target motion information.

[0239] Embodiment 22:

[0240] Based on the same application concept as the above method, the embodiment of the present application also proposes a coding device, which is applied to the coding end, such as Figure 8 As shown, it is a structural diagram of the device, and the device includes:

[0241] An acquisition module 81 is used to acquire a motion model of a current image block;

[0242] An establishing module 82, establishing a motion information list of the current image block according to the motion model;

[0243] A selection module 83 selects candidate motion information from the motion information list;

[0244] A determination module 84 determines the original motion information of the current image block according to the selected candidate motion information and the difference information of the current image block; and

[0245] Determining target motion information of the current image block according to the original motion information;

[0246] The encoding module 85 encodes the current image block according to the original motion information or the target motion information.

[0247] Embodiment 23:

[0248] The decoding end device provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to Fig. 9 As shown. It includes: a processor 91 and a machine-readable storage medium 92, wherein: the machine-readable storage medium 92 stores machine-executable instructions that can be executed by the processor 91; the processor 91 is used to execute the machine-executable instructions to implement the decoding method disclosed in the above example of this application.

[0249] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the decoding method disclosed in the above example of the present application can be implemented.

[0250] Embodiment 24:

[0251] The encoding end device provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to Fig.10 As shown. It includes: a processor 93 and a machine-readable storage medium 94, wherein: the machine-readable storage medium 94 stores machine-executable instructions that can be executed by the processor 93; the processor 93 is used to execute the machine-executable instructions to implement the encoding method disclosed in the above example of this application.

[0252] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the encoding method disclosed in the above example of the present application can be implemented.

[0253] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as CD, DVD, etc.), or similar storage medium, or a combination thereof.

[0254] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0255] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0256] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0257] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0258] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0259] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0260] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A decoding method, It is characterized in that Applied to a decoding end, the method comprises: Get the 2-parameter translation motion model of the current image block; Establishing a motion information list of the current image block according to the 2-parameter translation motion model; Select candidate motion information from the motion information list; Determining original motion information of the current image block according to the candidate motion information; Determine the original motion information as the center motion information, determine the edge motion information corresponding to the center motion information, and obtain the predicted value of the center motion information and the predicted value of the edge motion information; determine the target motion information of the current image block from the center motion information and the edge motion information according to the predicted value of the center motion information and the predicted value of the edge motion information; wherein obtaining the predicted value of the center motion information and the predicted value of the edge motion information includes: obtaining a template corresponding to the current image block, and obtaining the predicted value of the center motion information and the predicted value of the edge motion information according to the template; The encoded bit stream is decoded according to the original motion information or the target motion information.

2. The method according to claim 1, It is characterized in that The motion information list includes motion information of candidate image blocks corresponding to the current image block, and the motion information of the candidate image blocks includes a motion vector and a motion direction corresponding to a fixed reference frame; The candidate image block includes at least one of the following: an image block adjacent to the current image block in the current frame where the current image block is located; an image block not adjacent to the current image block in the current frame where the current image block is located; an image block in an adjacent frame of the current frame where the current image block is located; The image blocks in the adjacent frames of the current frame where the current image block is located include: a reference image block in the adjacent frames having the same position as the current image block, and adjacent image blocks of the reference image block.

3. The method according to claim 2, It is characterized in that The motion information list further includes default motion information including a zero vector and a zero reference frame index.

4. The method according to claim 2, It is characterized in that For the motion information of the plurality of candidate image blocks in the motion information list, determining the order of adding the motion information of each candidate image block according to the position relationship between each candidate image block and the current image block; According to the adding order, the motion information of each candidate image block is sequentially added to the motion information list until the amount of the motion information in the motion information list reaches a preset amount; Among them, when the motion information of each candidate image block is added to the motion information list in turn according to the adding order, for the motion information to be added, the motion information is compared with the motion information in the motion information list. If the two are the same, the motion information is not added to the motion information list; if the two are different, the motion information is added to the motion information list.

5. The method according to claim 1, It is characterized in that The obtaining of the template corresponding to the current image block includes: Determining a prediction value of the current image block by using original motion information of the current image block; A template corresponding to the current image block is obtained according to the predicted value of the current image block.

6. The method according to claim 1, It is characterized in that The current image block is a bidirectional inter-frame prediction block, the original motion information includes a first reference frame and a second reference frame corresponding to the current image block, a first motion vector corresponding to the first reference frame, and a second motion vector corresponding to the second reference frame; determining the original motion information as the center motion information, determining the edge motion information corresponding to the center motion information, obtaining the predicted value of the center motion information and the predicted value of the edge motion information; determining the target motion information of the current image block from the center motion information and the edge motion information according to the predicted value of the center motion information and the predicted value of the edge motion information, includes: Determine the first motion vector as a first center motion vector, and determine the second motion vector as a second center motion vector; determine a first edge motion vector corresponding to the first center motion vector, and determine a second edge motion vector corresponding to the second center motion vector; Determine a first initial reference block corresponding to the current image block from the first reference frame according to the first central motion vector, and determine a first prediction value of the first initial reference block by using a bilinear interpolation method; Determine a second initial reference block corresponding to the current image block from the second reference frame according to the second central motion vector, and determine a second prediction value of the second initial reference block by using a bilinear interpolation method; Determine a third initial reference block corresponding to the current image block from the first reference frame according to the first edge motion vector, and determine a third prediction value of the third initial reference block by using a bilinear interpolation method; Determine a fourth initial reference block corresponding to the current image block from the second reference frame according to the second edge motion vector, and determine a fourth prediction value of the fourth initial reference block by using a bilinear interpolation method; Based on the first prediction value, the second prediction value, the third prediction value and the fourth prediction value, a first target reference block and a second target reference block are searched, and the motion information of the first target reference block and the motion information of the second target reference block are determined as target motion information of the current image block.

7. The method according to claim 6, It is characterized in that The searching for the first target reference block and the second target reference block based on the first prediction value, the second prediction value, the third prediction value and the fourth prediction value comprises: Downsampling the first prediction value and the second prediction value, and determining a first similarity using the downsampled first prediction value and the downsampled second prediction value; downsampling the third prediction value and the fourth prediction value, and determining a second similarity using the downsampled third prediction value and the downsampled fourth prediction value; wherein the first similarity and the second similarity are SAD; A first target reference block and a second target reference block are determined according to the first similarity and the second similarity; wherein, if the first similarity is higher than the second similarity, the first initial reference block is determined as the first target reference block, and the second initial reference block is determined as the second target reference block; if the second similarity is higher than the first similarity, the third initial reference block is determined as the first target reference block, and the fourth initial reference block is determined as the second target reference block.

8. The method according to claim 1, It is characterized in that The original motion information includes a first original motion vector and a second original motion vector corresponding to the current image block; the target motion information includes a first target motion vector and a second target motion vector corresponding to the current image block; wherein: Determine a first target motion vector corresponding to the current image block according to the first original motion vector; determine a difference between the first original motion vector and the first target motion vector; and determine a second target motion vector corresponding to the current image block according to the second original motion vector and the difference.

9. A decoding terminal device, It is characterized in that include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Video image processing method and coder / decoder

    CN102215387A

  • Moving image encoding apparatus and decoding apparatus

    JP2009212667A