An image decoding method, an image encoding / decoding device, and a computer storage medium.

Abstract

This application proposes an image decoding method, an image encoding / decoding apparatus, and a computer storage medium. The image decoding method includes: acquiring the original motion vector of a block to be decoded; acquiring a first offset of the original motion vector using a preset motion vector thinning technique; adding the first offset to the original motion vector to obtain a thinned motion vector; acquiring a bidirectional prediction value of the thinned motion vector in response to the thinned motion vector being a bidirectional prediction motion vector; acquiring a second offset of the bidirectional prediction value using a preset prediction value thinning technique; obtaining a thinned prediction value based on the bidirectional prediction value and the second offset; and decoding the block to be decoded using the thinned prediction value to obtain a decoded block. This application can improve the efficiency of encoding and decoding.

Description

Technical Field

[0001] This application relates to the field of image encoding and decoding technology, and in particular to an image decoding method, an image encoding and decoding device, and a computer storage medium. Background Technology

[0002] Video image data is relatively large, so it is usually necessary to compress the video pixel data (RGB, YUV, etc.). The compressed data is called the video stream. The video stream is transmitted to the user's end via wired or wireless network for decoding and viewing. The entire video encoding process includes block partitioning, prediction, transform, quantization, and encoding.

[0003] During the long-term research and development process, the inventors of this application discovered that current image decoding methods still have certain limitations, which also affect the efficiency of encoding and decoding to some extent. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes an image decoding method, an image encoding / decoding device, and a computer storage medium.

[0005] To address the aforementioned technical problems, this application proposes an image decoding method, which includes:

[0006] Obtain the original motion vector of the block to be decoded;

[0007] The first offset of the original motion vector is obtained by using a preset motion vector refinement technique;

[0008] Add the first offset to the original motion vector to obtain the refined motion vector;

[0009] In response to the refined motion vector being a bidirectional predicted motion vector, the bidirectional predicted value of the refined motion vector is obtained;

[0010] The second offset of the bidirectional predicted value is obtained by using a preset predicted value refinement technique;

[0011] Based on the bidirectional prediction value and the second offset, a refined prediction value is obtained;

[0012] The refined prediction value is used to decode the block to be decoded to obtain the decoded block.

[0013] To address the aforementioned technical problems, this application proposes another image decoding method, which includes:

[0014] Obtain the original motion vector of the block to be decoded;

[0015] In response to the original motion vector being a bidirectional predicted motion vector, the bidirectional predicted value of the original motion vector is obtained;

[0016] Determine the forward and backward image sequence counting information based on the original motion vector;

[0017] In response to the forward and backward image sequence count information being different from the difference between the forward image sequence count and the current image sequence count, the block to be decoded is divided into at least one prediction sub-block.

[0018] Based on the original motion vector and the position of the predicted sub-block in the block to be decoded, the search area of ​​the predicted sub-block is determined;

[0019] Perform an integer pixel search process in the search area:

[0020] Obtain the first candidate offset, and determine the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset;

[0021] Obtain the distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset;

[0022] Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure;

[0023] Based on the bidirectional prediction value and the second offset, a refined prediction value is obtained;

[0024] The refined prediction value is used to decode the block to be decoded to obtain the decoded block.

[0025] To address the aforementioned technical problems, this application proposes another image decoding method, which includes:

[0026] Obtain the original motion vector of the block to be decoded;

[0027] In response to the original motion vector being a bidirectional predicted motion vector, the bidirectional predicted value of the original motion vector is obtained;

[0028] The block to be decoded is divided into at least one prediction sub-block;

[0029] Based on the original motion vector and the position of the predicted sub-block in the block to be decoded, the search area of ​​the predicted sub-block is determined;

[0030] Perform an integer pixel search process in the search area:

[0031] Obtain the first candidate offset, and determine the second candidate offset based on the first candidate offset;

[0032] Obtain the first reference template of the first candidate offset, the second reference template of the second candidate offset, and the current reference template of the predicted sub-block;

[0033] Obtain the first distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset;

[0034] Obtain the second distortion measure between the current reference template and the first reference template;

[0035] Obtain the third distortion measure between the current reference template and the second reference template;

[0036] A distortion measure is formed by using the first distortion measure, the second distortion measure, and the third distortion measure;

[0037] Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure;

[0038] Based on the bidirectional prediction value and the second offset, a refined prediction value is obtained;

[0039] The refined prediction value is used to decode the block to be decoded to obtain the decoded block.

[0040] To address the aforementioned technical problems, this application also proposes an image encoding / decoding apparatus, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the image decoding method described above.

[0041] To address the aforementioned technical problems, this application also proposes a computer storage medium for storing program data, which, when executed by a computer, is used to implement the aforementioned image decoding method.

[0042] Compared with existing technologies, the beneficial effects of this application are as follows: the image encoding / decoding device acquires the original motion vector of the block to be decoded; using a preset motion vector thinning technique, it acquires a first offset of the original motion vector; the original motion vector is added to the first offset to obtain a thinned motion vector; in response to the thinned motion vector being a bidirectional prediction motion vector, a bidirectional prediction value of the thinned motion vector is acquired; using a preset prediction value thinning technique, a second offset of the bidirectional prediction value is acquired; based on the bidirectional prediction value and the second offset, a thinned prediction value is obtained; the block to be decoded is decoded using the thinned prediction value to obtain a decoded block. This application can improve the efficiency of encoding and decoding. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] in:

[0045] Figure 1 This is a schematic diagram of an embodiment of the encoding / decoding process provided in this application;

[0046] Figure 2 This is a flowchart illustrating the first embodiment of the image decoding method provided in this application;

[0047] Figure 3 This is a flowchart illustrating the image decoding technology provided in this application;

[0048] Figure 4 This is a schematic diagram of the shape search based on template-refined motion vector technology provided in this application;

[0049] Figure 5 This is a schematic diagram of the template search based on template-refined motion vector technology provided in this application;

[0050] Figure 6 This is a schematic diagram of adjacent positions in the spatial domain based on the preset position refinement motion vector technology provided in this application;

[0051] Figure 7 This is a schematic diagram of the template provided in this application, which is based on the technique of refining motion vectors using a preset position template.

[0052] Figure 8 This is a schematic diagram of a candidate MVAP based on the preset position template refined motion vector technology provided in this application;

[0053] Figure 9 This is a flowchart illustrating the second embodiment of the image decoding method provided in this application;

[0054] Figure 10 This is a schematic diagram of sub-block pre-compensation based on the predicted value-refined motion vector technology provided in this application;

[0055] Figure 11 This is a schematic diagram of an embodiment of the motion vector refinement technique based on predicted values ​​for offsetting candidate points provided in this application;

[0056] Figure 12 This is a schematic diagram of an embodiment of the pixel-level candidate based on the predicted value refinement motion vector technique provided in this application;

[0057] Figure 13This is a flowchart illustrating the third embodiment of the image decoding method provided in this application;

[0058] Figure 14 This is a schematic diagram of the template area provided in this application for the technique of refining motion vectors based on predicted values;

[0059] Figure 15 This is a schematic diagram of the difference calculation process for the combined template provided in this application;

[0060] Figure 16 This is a flowchart illustrating the fourth embodiment of the image decoding method provided in this application;

[0061] Figure 17 This is a flowchart illustrating the fifth embodiment of the image decoding method provided in this application;

[0062] Figure 18 This is a schematic diagram of an embodiment of the image encoding and decoding apparatus provided in this application;

[0063] Figure 19 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0065] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0066] Please see Figure 1 , Figure 1 This is a schematic diagram of an embodiment of the encoding / decoding process provided in this application. Figure 1As shown, the overall encoding and decoding process is as follows: the encoding process starts from the input video frame and ends at the bitstream, while the decoding process starts from the bitstream and ends at the reconstructed frame. The dashed lines represent the common encoding and decoding processes, while the solid arrows indicating "bitstream → entropy decoding → inverse quantization & inverse transform" represent the decoding-specific processes. The remaining solid arrows represent the encoding-specific processes.

[0067] In video encoding, the most commonly used color encoding methods include YUV and RGB. This application uses the YUV color encoding method. Y represents luminance, which is the grayscale value of the image; U and V (i.e., Cb and Cr) represent chrominance, which describes the color and saturation of the image. Each Y luminance block corresponds to one Cb and one Cr chrominance block, and each chrominance block corresponds to only one luminance block. Taking a 4:2:0 sampling format as an example, an N*M block corresponds to a luminance block of size N*M, and the corresponding two chrominance blocks are both (N / 2)*(M / 2) in size, with the chrominance block being 1 / 4 the size of the luminance block. For a 4:4:4 sampling format, the luminance block and chrominance block are the same size.

[0068] Block partitioning: In video encoding, the input is a series of image frames. However, to encode a single frame, it needs to be divided into several LCUs (largest coding units). Then, each coding unit is recursively divided into CUs (coding units) of different sizes. Video encoding is performed using CUs as units. The smallest coding unit is called the SCU (smallest coding unit).

[0069] Intra-frame / Inter-frame prediction: Generally, the luminance and chrominance signal values ​​of adjacent pixels are quite similar and have a strong correlation. If the number of samples is used directly to represent luminance and chrominance information, there is a lot of spatial redundancy in the data. If redundant data is removed before encoding, the average number of bits per pixel will decrease, which means data compression is performed to reduce spatial redundancy.

[0070] Transformation: After the prediction of the current block is completed, the true value and the predicted value of the current block are subtracted to obtain a residual block. The residual block represents the difference between the true image and the predicted image of the current block. Then, a transformation is performed on the residual block, such as using DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform). Since most images have many flat regions and regions with slow content transformation, and the correlation between adjacent pixels is very strong, the transformation can reduce these correlations and transform the dispersed distribution of image energy in the spatial domain into a relatively concentrated distribution in the transform domain, thus removing spatial redundancy.

[0071] Quantization: Quantization is the process of mapping continuous signal values ​​to multiple discrete amplitude values, achieving a many-to-one mapping of signal values. After transformation, the transform coefficients of the residual data have a large range of values. Quantization can effectively reduce the range of signal values, thus achieving better compression. Because quantization discretizes continuous values ​​into various quantization intervals, it is the root cause of image distortion.

[0072] This application provides an image encoding and decoding method. Please refer to [link / reference needed] for details. Figure 2 and Figure 3 , Figure 2 This is a flowchart illustrating the first embodiment of the image decoding method provided in this application. Figure 3 This is a flowchart illustrating the image decoding technology provided in this application.

[0073] The image decoding method of this application is applied to an image encoding and decoding apparatus, wherein the image encoding and decoding apparatus of this application can be a server, a terminal device, or a system in which the server and the terminal device cooperate with each other. Accordingly, the various parts of the image encoding and decoding apparatus, such as various units, sub-units, templates, and sub-templates, can all be set in the server, all in the terminal device, or separately in the server and the terminal device.

[0074] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software templates, such as software or software templates used to provide distributed servers, or as a single software program or software template; no specific limitations are made here.

[0075] like Figure 3 As shown, the overall implementation process of the image decoding technology provided in this application specifically includes the following:

[0076] (1) DMVR (Decoder side motion vector refinement) application conditions: Determine whether the current CU (Coding Unit) meets the execution conditions of DMVR; if DMVR is executed, continue other processes, otherwise do not execute DMVR.

[0077] (2) Sub-block partitioning and pre-compensation: The current CU is partitioned and pre-compensation is performed. This has been introduced in the prior art and will not be repeated here.

[0078] (3) Integer pixel thinning and sub-pixel thinning: used to determine the specific position of the thinning adjustment prediction value in the pre-compensation result, thereby obtaining the thinning adjustment prediction value, which has been introduced in the prior art.

[0079] (4) Obtaining prediction based on thinning results: that is, obtaining thinning prediction values ​​from the pre-compensation results based on integer pixel thinning and sub-pixel thinning results.

[0080] like Figure 2 As shown, the specific steps are as follows:

[0081] Step S11: Obtain the original motion vector of the block to be decoded.

[0082] In this embodiment of the application, the image encoding and decoding device parses the transmitted bitstream information at the decoding end and obtains the original motion vector, i.e., the original MV, from the bitstream information.

[0083] Step S12: Use the preset motion vector refinement technology to obtain the first offset of the original motion vector.

[0084] In the embodiments of this application, the image encoding and decoding device uses motion vector thinning technology to thin the original MV and obtain the offset of the original MV.

[0085] Step S13: Add the first offset to the original motion vector to obtain the refined motion vector.

[0086] In this embodiment, the image encoding / decoding device uses a technique for thinning the motion vector (MV) based on a preset position. Under this technique, the MV used to obtain the predicted value is the original MV plus a preset thinning position. The preset position includes a preset distance and a prediction direction, as shown in the table below:

[0087] The table shows the preset distance based on the preset position refinement motion vector technology.

[0088] Distance Index 0 1 2 3 4 5 6 7 Preset distance 1 / 4-pel 1 / 2-pel 1-pel 2-pel 4-pel 8-pel 16-pel 32-pel

[0089] The table is based on the preset direction of the motion vector refinement technology at preset positions.

[0090] Direction Index 0 1 2 3 X-axis +(right) -(Left) N / A N / A Y-axis N / A N / A +(Next) -(superior)

[0091] In addition, regarding the source of MV for the preset position-based motion vector refinement technique, a candidate list of length 2 is constructed and one candidate is selected from it. In later techniques, the preset position-based motion vector refinement technique can sort several candidates based on template cost and retain only the two candidates with the lowest template cost, and select one candidate from them.

[0092] It should be noted that the techniques used in this application to refine motion vectors based on preset positions include, but are not limited to: UMVE (Ultimate Motion Vector Expression) and similar or extended techniques.

[0093] It is understandable that the motion vector refinement technology based on preset position can be used for unidirectional prediction and bidirectional prediction. When the motion vector refinement technology based on preset position can be used to refine motion vectors based on predicted values, it only includes the case where the candidate of the motion vector refinement technology based on preset position is bidirectional prediction.

[0094] It should be noted that the motion vector refinement techniques based on predicted values ​​used in this application include, but are not limited to, DMVR and similar or extended techniques.

[0095] Image encoding and decoding devices can also use a template-based technique to refine motion vectors (MVs). Under this template-based motion vector refinement technique, the final MV used to obtain the predicted value is the original MV plus the offset obtained by the template-based motion vector refinement technique.

[0096] It should be noted that the template-based techniques used in this application to refine MV include, but are not limited to, interTM and similar or extended techniques.

[0097] The refinement process based on template-based motion vector refinement is as follows: first, 30 hexagonal searches are performed, followed by one round of square searches, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the shape search based on template-refined motion vector technology provided in this application. Figure 4 Each cell represents a 1 / 4 pixel distance. During the search process, each round uses the result of the previous round as the starting point (the initial starting point is the original MV), and the search result for each round is the preset position within the search shape that has the minimum template cost.

[0098] The template used in the template-based motion vector refinement search process to calculate the template cost (shaded area, upper region width and left region height are 4) is as follows: Figure 5 As shown, Figure 5 This is a schematic diagram of the template search based on template-refined motion vector technology provided in this application. The template cost is the sum of absolute differences (SAD) between the current template and the reference template. The current template consists of the adjacent reconstructed pixels of the current coding block, and the reference template consists of the adjacent pixels of the reference block pointed to by the MV. In the current template image, the white area represents the current block, and the shaded area represents the current template. For the reference template, the white area represents the reference block, and the shaded area represents the reference template.

[0099] In addition, regarding the source of MV based on template-refined motion vector technology, a list containing several candidates is constructed, the candidates are rearranged based on template cost (only the top few candidates with the lowest cost are retained in the rearranged candidate list), and one candidate is selected from the rearranged candidate list.

[0100] It is understandable that template-based motion vector refinement technology can be used for unidirectional and bidirectional prediction. When template-based motion vector refinement technology can be used to refine motion vectors based on predicted values, it only includes the case where the candidate of template-based motion vector refinement technology is bidirectional prediction.

[0101] Step S14: In response to refining the motion vector into a bidirectional predicted motion vector, obtain the bidirectional predicted value of the refined motion vector.

[0102] In this embodiment, when the motion vector refinement technology based on a preset position and the motion vector refinement technology based on a template in step S13 are used for bidirectional prediction, the resulting refined motion vector is a bidirectional predicted motion vector, which meets the usage conditions of the motion vector refinement technology based on predicted values. At this point, the various processes of the motion vector refinement technology based on predicted values ​​can be implemented based on the refined motion vector.

[0103] In Embodiment 1 provided in this application, the image encoding and decoding device uses a technique based on preset position thinning motion vector technology + a technique based on predicted value thinning motion vector technology to achieve image decoding:

[0104] A candidate list for refining motion vectors based on preset positions is constructed: the list length is 2. Adjacent blocks F, G, C, A, and D are sequentially traversed, and their motion information is added to the candidate list (the added motion information must be usable, and deduplication is performed during the addition process), until the preset length is reached. If the preset length is not reached after traversal, temporal motion vectors and / or zero vectors are added. The positions of adjacent blocks are as follows: Figure 6 As shown, Figure 6 This is a schematic diagram of adjacent positions in the spatial domain based on the preset position refinement motion vector technology provided in this application.

[0105] Suppose the vector of the candidate list is {MV0, MV1}.

[0106] One motion vector refinement technique candidate based on a preset position is selected: At the decoding end, the selected candidate can be determined by the syntax of the motion vector refinement technique based on the preset position. Let's assume the selected candidate is MV0. At the encoding end, the best candidate can be selected using rate-distortion optimization techniques, and the corresponding syntax is written into the bitstream.

[0107] Motion vector refinement based on preset positions is performed: This refinement includes distance and direction. The decoder can directly decode the distance and direction indices from the syntax, thus determining the preset refinement value. The encoder uses rate-distortion optimization to select the optimal preset refinement value from the distance and direction candidates, and writes the corresponding syntax into the bitstream.

[0108] The motion vector refinement technique based on prediction values ​​is performed as follows: Assuming that the selected MV0 is bidirectional prediction and the POC of the forward and backward reference frames satisfies the symmetry relationship with respect to the current frame, the process of sub-block division, sub-block pre-compensation, integer pixel refinement, sub-pixel refinement, and obtaining the refinement prediction value is performed.

[0109] In Embodiment 2 provided in this application, the image encoding and decoding device uses a technique of refining motion vectors based on preset position templates and a technique of refining motion vectors based on predicted values ​​to achieve image decoding.

[0110] This embodiment is similar to Embodiment 1 above, except that the construction of the candidate list is different. The other steps, such as the candidate selection process based on the preset position refinement motion vector technology, the refinement process based on the preset position refinement motion vector technology, and the refinement process based on the preset position refinement motion vector technology, are the same as in Embodiment 1 above.

[0111] In this embodiment, the candidate list is constructed as follows:

[0112] (1) Traverse candidates and calculate template cost: Obtain available and valid candidates (i.e., the candidate is an inter-frame prediction mode, has been decoded, and does not duplicate the candidate in the previous order) in the order of spatial candidate SMVP, temporal candidate TMVP (TemporalMotion Vector Prediction), and historical candidate HMVP (History-based Motion Vector Prediction), and calculate the template cost of the candidate.

[0113] The template cost calculation uses the adjacent area of ​​the leftmost column and the topmost row as the template, such as... Figure 7 As shown, the template cost is the sum of absolute differences (SAD) between the current template and the reference template. Figure 7 This is a schematic diagram of the template provided in this application, which is based on the technique of refining motion vectors using a preset position template.

[0114] The airspace candidate SMVP includes ordinary SMVP and MVAP candidates. The ordinary airspace candidates are the same as the candidates at the five positions in Example 1 above, and in the same order. The MVAP candidates are the adjacent candidates around the current block (in order from bottom left to top right), such as... Figure 8 As shown, Figure 8 The small and medium grids are 4x4 blocks. The length of the left area is 2H (excluding the top left corner), and the width of the top area is 2W (excluding the top left corner). H and W are the height and width of the current block, respectively. Figure 8 This is a schematic diagram of a candidate MVAP based on the preset position template-based motion vector refinement technology provided in this application.

[0115] Among them, the temporal candidate TMVP is the candidate position of the current block center position in the temporal co-location image.

[0116] Among them, the historical candidate HMVP is a candidate from the time-domain candidate list (up to 8).

[0117] Two candidates with the lowest template cost are obtained from the above candidates and sorted in ascending order of template cost to form a candidate list.

[0118] In Embodiment 3 provided in this application, the image encoding and decoding device uses a technique based on template-based thinning motion vector technology and a technique based on predicted value thinning motion vector technology to achieve image decoding:

[0119] Candidate list construction based on template-refined motion vector technology: Following the skip / direc mode candidate list construction method, only the top 4 candidates are selected (the 4 candidates are the temporal domain candidate TMVP, bidirectional spatial domain candidate, backward spatial domain candidate, and forward spatial domain candidate from the skip / direc mode candidate list). The candidates are then deduplicated and integer-pixelized. Finally, these candidates are reordered based on template cost (from low to high). The template is similar to that in Example 2 above, except that the template area consists of 4 columns on the left and 4 rows on the top.

[0120] Selecting one candidate: At the decoding end, the candidate template-based motion vector technique can be determined through template-based refined motion vector technique syntax. At the encoding end, the best candidate template-based refined motion vector technique can be selected through rate-distortion optimization techniques, and the corresponding template-based refined motion vector technique syntax is written into the bitstream.

[0121] Template-based motion vector refinement technique: The candidate vectors are refined using a template-based refinement method.

[0122] Perform prediction-based motion vector refinement: Assuming the selected candidate is bidirectional prediction and its forward and backward reference frames satisfy a symmetric relationship with respect to the POC of the current frame, perform sub-block division, sub-block pre-compensation, integer pixel refinement, sub-pixel refinement, and obtain refinement prediction values ​​based on prediction-based motion vector refinement.

[0123] Step S15: Use the preset prediction value refinement technology to obtain the second offset of the bidirectional prediction value.

[0124] Step S16: Based on the bidirectional predicted values ​​and the second offset, obtain the refined predicted values.

[0125] Step S17: Use the refined prediction values ​​to decode the block to be decoded to obtain the decoded block.

[0126] In the embodiments of this application, the technical implementation process of steps S15 to S17 can refer to the relevant process of motion vector refinement technology based on predicted values, and will not be repeated here.

[0127] In this application, an image encoding / decoding device acquires the original motion vector of the block to be decoded; uses a preset motion vector thinning technique to acquire a first offset of the original motion vector; adds the first offset to the original motion vector to obtain a thinned motion vector; in response to the thinned motion vector being a bidirectional predicted motion vector, acquires a bidirectional predicted value of the thinned motion vector; uses a preset predicted value thinning technique to acquire a second offset of the bidirectional predicted value; based on the bidirectional predicted value and the second offset, obtains a thinned predicted value; and uses the thinned predicted value to decode the block to be decoded to obtain a decoded block. Through the above image decoding method, the predicted value of the motion vector candidates is thinned, thereby providing a more accurate thinning result and improving the image decoding effect.

[0128] Furthermore, in the current design of motion vector refinement technology based on predicted values, the Proof of Conformity (POC) of the candidate vector's forward and backward reference frames must satisfy a symmetrical relationship with respect to the current frame's POC in order to perform DMVR, i.e., POC1-POC. cur =POC cur –POC0, where POC0, POC1, and POC cur These are the forward reference frame, backward reference frame, and current frame of the candidate vector, respectively.

[0129] Specifically, in order to achieve the second technical effect mentioned above, the image decoding method of this application is further modified for DMVR application conditions, that is, it allows asymmetric references to perform motion vector refinement based on prediction values.

[0130] The image decoding method of this application allows POC0, POC1, and POC. curThe three do not need to satisfy a symmetrical relationship. In this case, it is necessary to process the integer-pixel thinning process based on the predicted value thinning motion vector technique. Please refer to [link to details]. Figure 9 , Figure 9 This is a flowchart illustrating the second embodiment of the image decoding method provided in this application.

[0131] like Figure 9 As shown, the specific steps are as follows:

[0132] Step S21: Determine the forward and backward image sequence counting information based on the original motion vector.

[0133] Step S22: In response to the forward and backward image sequence count information being different from the difference between the forward image sequence count and the current image sequence count, the block to be decoded is divided into at least one prediction sub-block.

[0134] In this embodiment of the application, if the forward and backward image sequence counting information of the original motion vector represents POC1-POC cur =POC cur –POC0 indicates that the image encoding / decoding device can directly use DMVR technology to refine the predicted values. The forward and backward image sequence count information of the original motion vectors is represented as POC1-POC. cur ≠POC cur –POC0 indicates that the image encoding / decoding device needs to scale one of the candidate offsets during the integer-pixel search process and / or the fractional-pixel search process to adapt to the asymmetric reference scene.

[0135] In POC1-POC cur ≠POC cur In the case of –POC0, the image encoding / decoding device performs sub-block partitioning: the W*H size encoding unit is divided into subW*subH sub-blocks, where subW=W<16? W:16, subH=H<16? H:16.

[0136] Step S23: Based on the original motion vector and the position of the predicted sub-block in the block to be decoded, determine the search area of ​​the predicted sub-block.

[0137] In this embodiment of the application, the image encoding and decoding device performs sub-block partitioning: sub-block pre-compensation: a schematic diagram of sub-block pre-compensation is shown below. Figure 10 As shown, Figure 10This is a schematic diagram of the sub-block pre-compensation technique based on predicted value-refined motion vectors provided in this application. Pre-compensation of the sub-block is performed based on the forward motion vector (MV0) and backward motion vector (MV1) of the coding unit, and the relative position of the sub-block within the CU. First, MV0 / MV1 is integer-pixelated to determine the integer-pixel reference position of the coding unit in the reference image. Then, the actual integer-pixel reference position of the sub-block in the reference image is determined according to its position relative to the current block. Next, the predicted value is obtained based on this integer-pixel reference position. When obtaining the predicted value, it is necessary to obtain not only the pixels of the predicted area (the size of the sub-block) but also pixels of a certain area outside the predicted area (…). Figure 1 Adjacent pixels in the interpolation filtering region, such as Figure 10 As shown. In addition, after obtaining the pixels in these areas, padding processing (image padding area) is also required based on the outermost pixels.

[0138] Step S24: Perform an integer pixel search process in the search area.

[0139] In this embodiment, the image encoding / decoding device performs sub-block prediction value adjustment: first, it performs integer pixel refinement search, then it performs sub-pixel refinement search, and based on the final refinement result, it determines the sub-block prediction value. The integer pixel search process is further explained in steps S25 and S26.

[0140] Step S25: Obtain the first candidate offset, and determine the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset.

[0141] Step S26: Obtain the distortion measure of the candidate offset pair consisting of the first candidate offset and the second candidate offset.

[0142] In this embodiment of the application, the image encoding / decoding device performs an integer pixel search:

[0143] Calculate the SAD (Sum of Absolute Difference) for 21 candidate refinement locations (between the corresponding forward and backward prediction blocks). Figure 11 As shown, the candidate refinement positions are as follows: Figure 11 The colored dots are shown in the middle. Among them, Figure 11 This is a schematic diagram of an embodiment of the motion vector refinement technique based on predicted values ​​for offsetting candidate points provided in this application.

[0144] Among them, the forward candidate refinement position and the backward candidate refinement position are mirror images of each other with respect to the starting point. For example, if the forward candidate position is (1, 0), then the backward candidate position is (-1, 0).

[0145] The search process involves using SAD gradient descent to search for the optimal SAD point among 21 candidates. Specifically, starting from the initial point, a maximum of three rounds of searching are performed. In each round, the search proceeds in four directions (up, down, left, and right) to find the optimal (minimum SAD) position, using the previous search result as the starting point.

[0146] In the embodiments of this application, in steps S24 to S26 above, the image encoding and decoding device needs to determine a candidate offset based on the prediction value-based motion vector refinement technique, and then perform scaling processing on the other candidate offset after determining a value based on the mirror relationship of the previously determined candidate offset.

[0147] The scaling processing scheme is mainly determined by the asymmetric reference scene determined by the sequential counting information of the forward and backward images, including but not limited to the following schemes: determining the backward thinning position based on the forward thinning position, determining the forward thinning position based on the backward thinning position, determining the thinning position of the far reference frame based on the thinning position of the near reference frame, and determining the thinning position of the near reference frame based on the thinning position of the far reference frame.

[0148] Specifically, the backward refinement position is determined based on the forward refinement position: that is, the forward refinement position is the refinement position based on the motion vector refinement technique based on the predicted value, and the backward refinement position is obtained by scaling the POC distance between reference frames, which can be expressed as the following formula:

[0149] offset1 = (POC1 - POC) cur ) / (POC0-POC cur )*offset0

[0150] Here, offset0 and offset1 represent the forward thinning position and the backward thinning position, respectively.

[0151] The forward refinement position is determined based on the backward refinement position: that is, the backward refinement position is the refinement position obtained by refining the motion vector based on the predicted value, and the forward refinement position is obtained by scaling the POC distance between reference frames, which can be expressed as follows:

[0152] offset0 = (POC0 - POC) cur ) / (POC1-POC cur )*offset1

[0153] The refinement position of the far reference frame is determined based on the refinement position of the near reference frame: that is, the refinement position of the prediction direction where the POC distance is closer in the forward and backward reference frames is based on the original DMVR refinement position, while the other direction is obtained by scaling the POC distance between the reference frames, which can be expressed as:

[0154] offset1 -x =(POC)1-x -POC cur ) / (POC x -POC cur )*offset x

[0155] Where x = 0 or 1, if the POC distance between the forward reference frame and the current frame is less than or equal to the POC distance between the backward reference frame and the current reference frame, then x = 0; otherwise, x = 1.

[0156] The refinement position of the near reference frame is determined based on the refinement position of the far reference frame: that is, the refinement position of the prediction direction with the larger POC distance between the forward and backward reference frames is based on the original DMVR refinement position, while the other direction is obtained by scaling the POC distance between the reference frames, which can be expressed as:

[0157] offset 1-x =(POC) 1-x -POC cur ) / (POC x -POC cur )*offset x

[0158] Where x = 0 or 1, if the POC distance between the forward reference frame and the current frame is greater than or equal to the POC distance between the backward reference frame and the current reference frame, then x = 0; otherwise, x = 1.

[0159] Furthermore, since the pixel precision in the codec is limited, it is necessary to perform precision processing on the refined positions obtained by scaling. Processing methods include, but are not limited to:

[0160] (1) Arbitrary preset precision: The scaled and refined position can be processed to the existing preset precision by rounding up, rounding down, rounding to the nearest integer, etc. The existing preset precision includes integer pixel precision and 1 / 4 pixel precision (1 / 4 pixel precision can be obtained by interpolation to obtain pixels at 1 / 4, 2 / 4, and 3 / 4 pixel precision positions).

[0161] (2) Fixed pixel precision: The scaled and refined position can be processed into a fixed integer pixel precision by rounding up, rounding down, rounding to the nearest integer, etc. Fixed precision includes, but is not limited to, integer pixel precision, 1 / 4 pixel precision, etc.

[0162] In Embodiment 4 provided in this application, the image encoding and decoding apparatus uses the following improved technique based on predicted value-refined motion vector technology to achieve image decoding:

[0163] Refinement position processing: Determine the refinement position of the far reference frame based on the refinement position of the near reference frame.

[0164] Precision processing: With fixed pixel precision, the scaled and refined positions are rounded to integer pixel precision.

[0165] Assuming the forward reference frame's POC0 is 1, the backward reference frame's POC1 is 8, and the current reference frame's POC... cur Since the value is 3, the forward reference frame is relatively close to the current reference frame. Assuming the refinement position of the forward reference frame is offset0 = 1, then the refinement position of the backward reference frame is offset1 = (POC1 - POC). cur ) / (POC0-POC cur )*offset0=(8-3) / (1-3)*1=-2.5, rounding offset1 gives -2.

[0166] Step S27: Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure.

[0167] In this embodiment of the application, the image encoding and decoding device can determine the final offset obtained from the integer pixel search process according to the integer pixel thinning technique based on the predicted value thinning motion vector technique.

[0168] Furthermore, the image encoding and decoding device can also apply the scaling relationship determined in the above process to the pixel sub-process to obtain the scaled pixel offset. This is mainly reflected in the fact that the scaling relationship and scaling coefficient need to be added to the calculation expression obtained from the error surface model, which will not be elaborated here.

[0169] The pixel-by-pixel search process based on the predicted value-refined motion vector technique is as follows:

[0170] Pixel-level search condition: If the best integer-pixel candidate position is within a 1-pixel range, such as... Figure 12 If the color candidate points are shown, then a pixel-by-pixel search is performed. Figure 12 This is a schematic diagram of an embodiment of the pixel candidate based on the predicted value refinement motion vector technique provided in this application.

[0171] Pixel-by-pixel search process: The pixel-by-pixel search is based on error surface calculation. This search process assumes that local pixel values ​​are continuous, thereby establishing the SAD cost surface, i.e., the error surface model. The calculation expression for the pixel-by-pixel offset can be obtained from the error surface model as follows:

[0172] deltaX = (SAD) left -SAD right ) / ((SAD left +SAD right –2*SAD center )*2)

[0173] deltaY = (SAD) top -SAD bottom ) / ((SAD top +SAD bottom –2*SAD center )*2)

[0174] Here, deltaX and deltaY are the refined results of the pixel-level search, and SADcenter, SADleft, SADright, SADtop, and SADbottom are the best integer pixel candidate position and the SAD to its left, right, top, and bottom, respectively. Similarly, for the forward and candidate prediction values, the pixel-level search results are mirror images of each other, that is, the forward pixel-level search results are deltaX and deltaY, and the backward pixel-level search results are -deltaX and -deltaY.

[0175] Specifically, the image decoding method of this application continues to modify the integer pixel thinning in the motion vector thinning technique based on predicted values, that is, to enhance the integer pixel thinning process based on reconstructed values.

[0176] In the current design of motion vector thinning technology based on prediction values, the thinning prediction value is determined by minimizing the difference between the forward prediction value and the backward prediction value. However, the lack of accurate pixels (referring to reconstructed pixels) as a reference may lead to inaccurate thinning results.

[0177] This application designs a difference calculation method that combines templates for integer pixel thinning. Specifically, when calculating the difference between forward and backward predicted values, adjacent pixels are used as templates. Therefore, the difference calculation includes the difference between forward and backward predicted values ​​and the difference between the templates.

[0178] Please refer to the details. Figure 13 and Figure 14 , Figure 13 This is a flowchart illustrating the third embodiment of the image decoding method provided in this application. Figure 14 This is a schematic diagram of the template area provided in this application for the technique of refining motion vectors based on predicted values.

[0179] like Figure 13 As shown, the specific steps are as follows:

[0180] Step S31: Obtain the first reference template of the first candidate offset, the second reference template of the second candidate offset, and the current reference template of the predicted sub-block.

[0181] In the embodiments of this application, such as Figure 14As shown, the reference template (the white area of ​​the current template is the current block, and the white area of ​​the reference template is the reference block) is as follows, including but not limited to: (1) using only the pixels that can be obtained from the area above the block; (2) using only the pixels that can be obtained from the area to the left of the block; (3) using the pixels that can be obtained from the areas above and to the left of the block; (4) using the pixels that can be obtained from the areas above, to the left, and to the upper left corner of the block (L-shaped template area).

[0182] Step S32: Obtain the first distortion measure composed of the first candidate offset and the second candidate offset.

[0183] Step S33: Obtain the second distortion measure between the current reference template and the first reference template.

[0184] Step S34: Obtain the third distortion measure between the current reference template and the second reference template.

[0185] Step S35: Use the first distortion measure, the second distortion measure and the third distortion measure to form a distortion measure.

[0186] In this embodiment of the application, the image encoding and decoding device determines the distortion measure of three different objects through the above steps, and finally combines them into the final distortion measure. By introducing the current reference template, the accurate reconstructed pixels are used as a reference to improve the accuracy of the thinning result.

[0187] It should be noted that the distortion measures used in this application include, but are not limited to: SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSD (Sum of Squared Difference), MSE (Mean Squared Error), PSNR (Peak Signal to Noise Ratio), etc.

[0188] In Embodiment 5 provided in this application, the image encoding and decoding apparatus uses the following improved technique based on predicted value-refined motion vector technology to achieve image decoding:

[0189] like Figure 15As shown, in this embodiment, the absolute difference between the forward and backward predicted values ​​is SAD1 (i.e., the SAD between the forward reference block and the backward reference block), the absolute difference between the forward template is SAD2 (i.e., the SAD between the current template and the forward reference template), and the absolute difference between the backward template is SAD3 (i.e., the SAD between the current template and the backward reference template). The final difference is a combination of SAD1, SAD2, and SAD3, such as w1*SAD1+w2*SAD2+w3*SAD3, where w1, w2, and w3 are the weighting coefficients of the three.

[0190] Please see Figure 16 , Figure 16 This is a flowchart illustrating the fourth embodiment of the image decoding method provided in this application.

[0191] like Figure 16 As shown, the specific steps are as follows:

[0192] Step S41: Obtain the original motion vector of the block to be decoded.

[0193] Step S42: In response to the original motion vector being a bidirectional predicted motion vector, obtain the bidirectional predicted value of the original motion vector.

[0194] Step S43: Determine the forward and backward image sequence counting information based on the original motion vector.

[0195] Step S44: In response to the forward and backward image sequence count information being different from the difference between the forward image sequence count and the current image sequence count, the block to be decoded is divided into at least one prediction sub-block.

[0196] Step S45: Determine the search area of ​​the predicted sub-block based on the original motion vector and the position of the predicted sub-block in the block to be decoded.

[0197] Step S46: Perform an integer pixel search process in the search area.

[0198] Step S47: Obtain the first candidate offset, and determine the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset.

[0199] Step S48: Obtain the distortion measure of the candidate offset pair consisting of the first candidate offset and the second candidate offset.

[0200] Step S49: Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure.

[0201] Step S410: Based on the bidirectional predicted values ​​and the second offset, obtain the refined predicted values.

[0202] Step S411: Use the refined prediction values ​​to decode the block to be decoded to obtain the decoded block.

[0203] In the embodiments of this application, the image encoding and decoding device implements an asymmetric reference scheme based on the existing prediction value-based motion vector refinement technology, which will not be described in detail here.

[0204] Please see Figure 17 , Figure 17 This is a flowchart illustrating the fifth embodiment of the image decoding method provided in this application.

[0205] like Figure 17 As shown, the specific steps are as follows:

[0206] Step S51: Obtain the original motion vector of the block to be decoded.

[0207] Step S52: In response to the original motion vector being a bidirectional predicted motion vector, obtain the bidirectional predicted value of the original motion vector.

[0208] Step S53: Divide the block to be decoded into at least one prediction sub-block.

[0209] Step S54: Determine the search area of ​​the predicted sub-block based on the original motion vector and the position of the predicted sub-block in the block to be decoded.

[0210] Step S55: Perform an integer pixel search process in the search area.

[0211] Step S56: Obtain the first candidate offset and determine the second candidate offset based on the first candidate offset.

[0212] Step S57: Obtain the first reference template of the first candidate offset, the second reference template of the second candidate offset, and the current reference template of the predicted sub-block.

[0213] Step S58: Obtain the first distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset.

[0214] Step S59: Obtain the second distortion measure between the current reference template and the first reference template.

[0215] Step S510: Obtain the third distortion measure between the current reference template and the second reference template.

[0216] Step S511: Use the first distortion measure, the second distortion measure and the third distortion measure to form a distortion measure.

[0217] Step S512: Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure.

[0218] Step S513: Based on the bidirectional predicted values ​​and the second offset, obtain the refined predicted values.

[0219] Step S514: Use the refined prediction values ​​to decode the block to be decoded to obtain the decoded block.

[0220] In this embodiment, the image encoding and decoding device implements an integer pixel thinning scheme based on the prediction value thinning motion vector technology on the basis of the existing prediction value thinning motion vector technology, which will not be described in detail here.

[0221] Correspondingly, the improved syntax based on the predicted value-refined motion vector technique of this application includes, but is not limited to, the following three aspects:

[0222] (1) Switch syntax: used to express whether the scheme proposed in this application is enabled in the codec. The switch syntax can be transmitted in syntax structures including but not limited to: Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), coding unit, etc.

[0223] (2) Scheme syntax: Scheme syntax is used to describe whether to execute a scheme (whether to execute the motion vector refinement technique based on prediction values ​​is determined by the application conditions, so there is no need to describe the relevant syntax of whether to execute the motion vector refinement technique scheme based on prediction values) and how to execute the scheme, including but not limited to the refinement difference syntax of candidate motion vector refinement technique based on prediction values, refinement template syntax of motion vector refinement technique based on prediction values, etc.

[0224] Refinement Syntax Based on Predicted Value Refinement of Motion Vectors: This syntax is needed to express which method to choose when it is necessary to select between the original method of refining difference calculation based on predicted value refinement of motion vectors and the method of difference calculation based on predicted value refinement of motion vectors designed in this proposal.

[0225] Predictive value-based motion vector refinement template syntax: When the difference calculation method of predictive value-based motion vector refinement technology can be selected among multiple templates, it is used to select the template to use.

[0226] (3) Syntactic coding: Syntactic binarization methods include, but are not limited to, unary codes, truncated unary codes, truncated Rice codes, signed fixed-length codes, unsigned fixed-length codes, exponential Golomb codes, etc. Syntactic coding methods include, but are not limited to, high-entropy coding, bypass coding, etc. The coding methods are described in the standard text as descriptors. For specific meanings, please refer to the corresponding standard text.

[0227] In Embodiment 6 provided in this application, the image encoding / decoding apparatus uses the following improved syntax based on the predicted value-refined motion vector technique to implement image decoding:

[0228] In this embodiment, only the motion vector refinement technology based on preset position and the motion vector refinement technology based on template are implemented, while the motion vector refinement technology based on predicted value is supported.

[0229] In this embodiment, the switch syntax `sps_edmvr_enable` is used to express whether the motion vector refinement technique based on predicted values ​​is enabled, as shown in the table below. When `sps_edmvr_enable = 0`, the technique is not enabled; when `sps_edmvr_enable = 1`, the technique is enabled.

[0230]

[0231] Decoding process: When sps_edmvr_enable is 1, and the current encoding unit UmveFlag and InterTMFlag are 1.

[0232] The value of DmvrFlag is 0 if one of the following conditions is met; otherwise, the value of DmvrFlag is 1.

[0233] 1) The value of DmvrEnableFlag is 0.

[0234] 2) The prediction reference mode of the current sub-block is not 'PRED_List01'.

[0235] 3) The InterPcFlag value of the current prediction unit is 1.

[0236] 4) The height or width of the current prediction cell is less than 8.

[0237] 5) The current prediction cell has a height of 4 and a width of 8.

[0238] 6) The current prediction cell has a height of 8 and a width of 4.

[0239] 7) The image with reference index RefIndexL0 in the reference image queue 0 of the current sub-block and the image with reference index RefIndexL1 in the reference image queue 1 are both located before or after the current image display order.

[0240] 8) The image distance index of the image with reference index RefIndexL0 in the reference image queue 0 of the current sub-block is DistanceIndexRef0, the image distance index of the image with reference index RefIndexL1 in the reference image queue 1 of the current sub-block is DistanceIndexRef1, and the distance index of the current image is DistanceIndexCur. Abs(DistanceIndexRef0-DistanceIndexCur) and Abs(DistanceIndexRef1-DistanceIndexCur) are not equal.

[0241] 9) The image with reference index refIndexL0 in the reference image queue 0 of the current sub-block and the image with reference index refIndexL1 in the reference image queue 1 are the same reference image, and the motion vectors of L0 and L1 are the same.

[0242] When DmvrFlag is 1, it means that the technique of refining motion vectors based on predicted values ​​can be performed.

[0243] In Embodiment 7 provided in this application, the image encoding / decoding apparatus uses the following improved syntax based on the predicted value-refined motion vector technique to implement image decoding:

[0244] In this embodiment, the motion vector refinement technology based on preset position and the motion vector refinement technology based on template support the motion vector refinement technology based on predicted value. Furthermore, the motion vector refinement technology based on predicted value supports the asymmetric reference scheme and the whole-pixel refinement scheme based on the motion vector refinement technology based on predicted value (replacing the existing scheme and using the template in the above embodiment 5 by default, without the need for related scheme syntax).

[0245] Similarly, sps_edmvr_enable is used for the on / off syntax in this scheme.

[0246] Decoding process: When sps_edmvr_enable is 1, prediction modes based on predicted value-refined motion vector techniques are supported. Regarding the conditions for using predicted value-refined motion vector techniques, the following restrictions are removed:

[0247] 1) The image with reference index RefIndexL0 in the reference image queue 0 of the current sub-block and the image with reference index RefIndexL1 in the reference image queue 1 are both located before or after the current image display order.

[0248] 2) The image distance index of the image with reference index RefIndexL0 in the reference image queue 0 of the current sub-block is DistanceIndexRef0. The image distance index of the image with reference index RefIndexL1 in the reference image queue 1 of the current sub-block is DistanceIndexRef1. The distance index of the current image is DistanceIndexCur. Abs(DistanceIndexRef0-DistanceIndexCur) and Abs(DistanceIndexRef1-DistanceIndexCur) are not equal.

[0249] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0250] To implement the above image decoding method, this application also proposes an image encoding and decoding apparatus, which can be found in detail below. Figure 18 , Figure 18 This is a schematic diagram of an embodiment of the image encoding and decoding apparatus provided in this application.

[0251] The image encoding / decoding device 500 of this embodiment includes a processor 51, a memory 52, an input / output device 53, and a bus 54.

[0252] The processor 51, memory 52, and input / output device 53 are respectively connected to the bus 54. The memory 52 stores program data, and the processor 51 is used to execute the program data to implement the image decoding method described in the above embodiments.

[0253] In this embodiment, processor 51 can also be referred to as a CPU (Central Processing Unit). Processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 51 can be any conventional processor.

[0254] This application also provides a computer storage medium; please refer to the following: Figure 19 , Figure 19 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 600 stores a computer program 61, which, when executed by a processor, is used to implement the image decoding method of the above embodiment.

[0255] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0256] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image decoding method characterized by, The image decoding method includes: Obtain the original motion vector of the block to be decoded; The first offset of the original motion vector is obtained by using a preset motion vector refinement technique; Add the first offset to the original motion vector to obtain the refined motion vector; In response to the refined motion vector being a bidirectional predicted motion vector, the bidirectional predicted value of the refined motion vector is obtained; The second offset of the bidirectional predicted value is obtained by using a preset predicted value refinement technique; Based on the bidirectional prediction value and the second offset, a refined prediction value is obtained; The refined prediction value is used to decode the block to be decoded to obtain the decoded block.

2. The image decoding method according to claim 1, characterized in that, wherein The first offset is the offset by which the original motion vector is refined based on the template.

3. The image decoding method according to claim 1, characterized in that, wherein, The first offset is the offset by which the original motion vector is refined based on a preset position.

4. The image decoding method according to claim 1, characterized in that, The step of obtaining the second offset of the bidirectional prediction value using a preset prediction value refinement technique includes: Determine the forward and backward image sequence counting information based on the original motion vector; In response to the forward and backward image sequence count information being: when the difference between the forward image sequence count and the current image sequence count is the same as the difference between the backward image sequence count and the current image sequence count, the block to be decoded is divided into at least one prediction sub-block; Based on the original motion vector and the position of the predicted sub-block in the block to be decoded, the search area of ​​the predicted sub-block is determined; Perform an integer pixel search process in the search area: Obtain the first candidate offset, and determine the second candidate offset based on the first candidate offset; Obtain the distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset; Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure.

5. The image decoding method according to claim 1, characterized in that, The step of obtaining the second offset of the bidirectional prediction value using a preset prediction value refinement technique includes: Determine the forward and backward image sequence counting information based on the original motion vector; In response to the forward and backward image sequence count information being different from the difference between the forward image sequence count and the current image sequence count, the block to be decoded is divided into at least one prediction sub-block. Based on the original motion vector and the position of the predicted sub-block in the block to be decoded, the search area of ​​the predicted sub-block is determined; Perform an integer pixel search process in the search area: Obtain the first candidate offset, and determine the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset; Obtain the distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset; Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure.

6. The image decoding method according to claim 5, characterized in that, The step of determining the second offset corresponding to the minimum distortion measure based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process includes: If the second integer pixel offset corresponding to the minimum distortion measure obtained from the integer pixel search process is less than a preset range, a sub-pixel search process is performed in the search area based on the second integer pixel offset: The second sub-pixel offset is calculated using the error curve model and the sequential counting information of the forward and backward images. The second offset is determined based on the second integer pixel offset and the second fractional pixel offset.

7. The image decoding method according to claim 5, characterized in that, The first candidate offset is a forward candidate offset, and the second candidate offset is a backward candidate offset; The step of obtaining the first candidate offset and determining the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset includes: The forward candidate offset is obtained according to the preset prediction value refinement technique; Based on the forward and backward image sequence counting information, determine the forward image sequence count and the backward image sequence count; The first scaling relationship is determined based on the forward image sequence count, the backward image sequence count, and the current image sequence count; The forward candidate offset is scaled using the first scaling relationship to obtain the backward candidate offset.

8. The image decoding method according to claim 5, characterized in that, The first candidate offset is a backward candidate offset, and the second candidate offset is a forward candidate offset; The step of obtaining the first candidate offset and determining the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset includes: The backward candidate offset is obtained according to the preset prediction value refinement technique; Based on the forward and backward image sequence counting information, determine the forward image sequence count and the backward image sequence count; The second scaling relationship is determined based on the forward image sequence count, the backward image sequence count, and the current image sequence count; The backward candidate offset is scaled using the second scaling relationship to obtain the forward candidate offset.

9. The image decoding method according to claim 5, characterized in that, The step of obtaining the first candidate offset and determining the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset includes: Based on the forward and backward image sequence counting information, determine the forward image sequence count and the backward image sequence count; Obtain the first difference between the forward image sequence count and the current image sequence count; Obtain the second difference between the backward image sequence count and the current image sequence count; The first image sequence count is determined based on the smaller of the first difference and the second difference; The first candidate offset corresponding to the sequential counting of the first image is obtained according to the preset prediction value refinement technique; The third scaling relationship is determined based on the forward image sequence count, the backward image sequence count, and the current image sequence count; The first candidate offset is scaled using the third scaling relationship to obtain the second candidate offset.

10. The image decoding method according to claim 5, characterized in that, The step of obtaining the first candidate offset and determining the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset includes: Based on the forward and backward image sequence counting information, determine the forward image sequence count and the backward image sequence count; Obtain the first difference between the forward image sequence count and the current image sequence count; Obtain the second difference between the backward image sequence count and the current image sequence count; The first image sequence count is determined based on the larger of the first difference and the second difference; The first candidate offset corresponding to the sequential counting of the first image is obtained according to the preset prediction value refinement technique; The fourth scaling relationship is determined based on the forward image sequence count, the backward image sequence count, and the current image sequence count; The first candidate offset is scaled using the fourth scaling relationship to obtain the second candidate offset.

11. The image decoding method according to any one of claims 5 to 10, characterized in that, After obtaining the first candidate offset and determining the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset, the image decoding method further includes: The second candidate offset is processed according to a preset pixel precision to obtain the third candidate offset; The preset pixel precision includes whole pixel precision and / or 1 / 4 pixel precision.

12. The image decoding method according to claim 5, characterized in that, The step of obtaining the distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset includes: Obtain the first reference template of the first candidate offset, the second reference template of the second candidate offset, and the current reference template of the predicted sub-block; Obtain a first distortion measure composed of the first candidate offset and the second candidate offset; Obtain the second distortion measure between the current reference template and the first reference template; Obtain the third distortion measure between the current reference template and the second reference template; The distortion measure is composed of the first distortion measure, the second distortion measure, and the third distortion measure.

13. An image decoding method, characterized by, The image decoding method includes: Obtain the original motion vector of the block to be decoded; In response to the original motion vector being a bidirectional predicted motion vector, the bidirectional predicted value of the original motion vector is obtained; Determine the forward and backward image sequence counting information based on the original motion vector; In response to the forward and backward image sequence count information being different from the difference between the forward image sequence count and the current image sequence count, the block to be decoded is divided into at least one prediction sub-block. Based on the original motion vector and the position of the predicted sub-block in the block to be decoded, the search area of ​​the predicted sub-block is determined; Perform an integer pixel search process in the search area: Obtain the first candidate offset, and determine the second candidate offset based on the forward and backward image sequence counting information and the first candidate offset; Obtain the distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset; Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure; Based on the bidirectional prediction value and the second offset, a refined prediction value is obtained; The refined prediction value is used to decode the block to be decoded to obtain the decoded block.

14. An image decoding method, characterized by, The image decoding method includes: Obtain the original motion vector of the block to be decoded; In response to the original motion vector being a bidirectional predicted motion vector, the bidirectional predicted value of the original motion vector is obtained; The block to be decoded is divided into at least one prediction sub-block; Based on the original motion vector and the position of the predicted sub-block in the block to be decoded, the search area of ​​the predicted sub-block is determined; Perform an integer pixel search process in the search area: Obtain the first candidate offset, and determine the second candidate offset based on the first candidate offset; Obtain the first reference template of the first candidate offset, the second reference template of the second candidate offset, and the current reference template of the predicted sub-block; Obtain the first distortion measure of the candidate offset pair composed of the first candidate offset and the second candidate offset; Obtain the second distortion measure between the current reference template and the first reference template; Obtain the third distortion measure between the current reference template and the second reference template; A distortion measure is formed by using the first distortion measure, the second distortion measure, and the third distortion measure; Based on the distortion measure of all candidate offset pairs obtained from the integer pixel search process, determine the second offset corresponding to the minimum distortion measure; Based on the bidirectional prediction value and the second offset, a refined prediction value is obtained; The refined prediction value is used to decode the block to be decoded to obtain the decoded block.

15. An image encoding / decoding apparatus, characterized in that, The image encoding / decoding device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the image decoding method as described in any one of claims 1 to 14.

16. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the image decoding method as described in any one of claims 1 to 14.

Images