Video decoding method, apparatus, device, and storage medium

Abstract

This disclosure provides a video decoding method, apparatus, device, and storage medium. When decoding the current block, the decoding side first determines the first motion information of the current block, and then improves the first motion information based on the motion information of the reference image of the current block to obtain second motion information. That is, in the embodiments of this application, when improving the first motion information, the motion information of the reference image is taken into consideration, and the first motion information is effectively improved to obtain accurate second motion information. As a result, when determining the predicted value of the current block based on this accurate second motion information, the prediction accuracy of the current block can be increased, and the video decoding performance can be further improved.

Description

Technical Field

[0001] This disclosure relates to the field of video encoding and decoding technologies, and in particular, to a video decoding method, apparatus, device, and storage medium.

Background Art

[0002] Digital video technology can be incorporated into various video devices such as digital TVs, smartphones, computers, e-readers, or video players. With the development of video technology, the amount of data contained in video data is increasing. To facilitate the transmission of video data, video devices can execute video compression technology to transmit or store video data more effectively.

[0003] Since there is temporal or spatial redundancy in video, redundancy in the video can be removed or reduced by prediction to improve compression efficiency. During prediction, to improve decoding accuracy, the decoder side improves the motion information determined by decoding. However, the current motion information improvement method has insufficient improvement effect and inaccurate prediction on the decoder side, thus affecting the decoding performance of the video.

Summary of the Invention

Means for Solving the Problems

[0004] Embodiments of the present application provide a video decoding method, apparatus, device, and storage medium, which can improve the improvement effect of motion information and enhance the prediction accuracy of the current block and the decoding performance of the video.

[0005] In a first aspect, the present application provides a video decoding method applied to a decoder, the method including: determining first motion information of a current block; improving the first motion information based on motion information of a reference image of the current block to obtain second motion information of the current block; determining a predicted value of the current block based on the second motion information.

[0006] In a second aspect, the present application provides a video decoding device used to carry out the method in the first aspect or each embodiment thereof. Specifically, the device includes a functional unit for carrying out the method in the first aspect or each embodiment thereof.

[0007] A third aspect provides a video decoder, including a processor and memory. The memory is used to store computer programs, and the processor is used to perform the methods of the first aspect or each embodiment thereof by calling and executing the computer programs stored in the memory.

[0008] A fourth aspect provides a video encoding and decoding system, which includes a video encoder and a video decoder. The video decoder is used to carry out the method in the first aspect or each embodiment thereof.

[0009] In the fifth embodiment, a chip is provided and used to implement the method in the first embodiment. Specifically, the chip includes a processor for calling and executing a computer program from memory, thereby causing the device on which the chip is installed to execute the method in the first embodiment.

[0010] In the sixth aspect, a computer-readable storage medium is provided, which is used to store a computer program, and the computer program causes the computer to execute the method in the first aspect.

[0011] In the seventh aspect, a computer program product is provided, which includes computer program instructions, the computer program instructions causing a computer to execute the method in the first aspect.

[0012] The eighth aspect provides a method according to any aspect of the first to second aspects or each embodiment thereof.

[0013] Based on the above technical solution, when the decoding side decodes the current block, it first determines the first motion information of the current block, and then improves the first motion information based on the motion information of the reference image of the current block to obtain second motion information. That is, in the embodiment of the present application, when improving the first motion information, the motion information of the reference image is taken into consideration, and the first motion information is effectively improved to obtain accurate second motion information. As a result, when determining the predicted value of the current block based on this accurate second motion information, the prediction accuracy of the current block can be increased, and the video decoding performance can be further improved. [Brief explanation of the drawing]

[0014] [Figure 1] This is a schematic block diagram of a video encoding and decoding system according to an embodiment of the present invention. [Figure 2] This is a schematic block diagram of a video encoder according to an embodiment of the present invention. [Figure 3] This is a schematic block diagram of a video decoder according to an embodiment of the present invention. [Figure 4] This is a schematic diagram of the GOP structure. [Figure 5] This is a schematic diagram of the CU partitioning. [Figure 6] This is a schematic diagram of the spatial and temporal domain blocks. [Figure 7] This is a schematic diagram for deriving time-domain motion information. [Figure 8] This is a schematic diagram of the SbTMVP principle. [Figure 9] This is a schematic diagram illustrating the principle of MMVD. [Figure 10A] This is a schematic diagram of the affine principle. [Figure 10B] This is a schematic diagram of another principle of affine. [Figure 11] This is a schematic diagram of the weighting. [Figure 12] This is a schematic diagram illustrating the principle of DMVR. [Figure 13] This is a schematic diagram of template matching. [Figure 14]It is a diagram showing the flow of a video decoding method according to an embodiment of the present application. [Figure 15] It is a diagram showing a current block and a reference block. [Figure 16] It is a diagram showing the motion of a current block and two reference blocks. [Figure 17A] It is a diagram showing the motion when the motion information of the current block is larger than the motion information of the reference block. [Figure 17B] It is a diagram showing the motion when the motion information of the current block is smaller than the motion information of the reference block. [Figure 18A] It is a diagram showing an example of another motion when the motion information of the current block is smaller than the motion information of the reference block. [Figure 18B] It is a diagram showing an example of another motion when the motion information of the current block is larger than the motion information of the reference block. [Figure 19A] It is a diagram showing an example of motion information search. [Figure 19B] It is a diagram showing another example of motion information search. [Figure 19C] It is a diagram showing yet another example of motion information search. [Figure 20] It is a diagram showing an example when different motion information exists in the reference block. [Figure 21] It is a schematic block diagram of a video decoding apparatus according to an embodiment of the present application. [Figure 22] It is a schematic block diagram of an electronic device according to an embodiment of the present application.

Modes for Carrying Out the Invention

[0015] This invention can be applied to the fields of image coding and decoding, video coding and decoding, hardware video coding and decoding, dedicated circuit video coding and decoding, and real-time video coding and decoding. For example, the solution of this invention can be combined with audio video coding standards (AVS), such as the H.264 / audio video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. Alternatively, the present invention may be used in conjunction with other proprietary or industry standards, including ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, and ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), and including Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions. It is understood that the present invention is not limited to any particular coding or decoding standard or technique.

[0016] To facilitate understanding, the video coding and decoding system according to the embodiment of this application will first be described with reference to Figure 1.

[0017] Figure 1 is a schematic block diagram of a video encoding / decoding system according to an embodiment of the present application. Note that Figure 1 is merely an example, and the video encoding / decoding system of the embodiment of the present application is not limited to that shown in Figure 1. As shown in Figure 1, the video encoding / decoding system 100 includes an encoding device 110 and a decoding device 120. Here, the encoding device is used to encode (understand as compressing) video data to generate a bitstream and transmit the bitstream to the decoding device. The decoding device decodes the bitstream generated by the encoding device and obtains the decoded video data.

[0018] In the embodiments of the present application, the encoding device 110 is understood to be a device having video encoding capabilities, and the decoding device 120 is understood to be a device having video decoding capabilities. That is, the embodiments of the present application include a broader range of devices than the encoding device 110 and the decoding device 120, including, for example, smartphones, desktop computers, mobile computing devices, notebook computers (e.g., laptops), tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, and the like.

[0019] In some embodiments, the encoding device 110 may transmit the encoded video data (e.g., a bitstream) to the decoding device 120 via channel 130. Channel 130 may include one or more media and / or devices capable of transmitting the encoded video data from the encoding device 110 to the decoding device 120.

[0020] In one example, channel 130 includes one or more communication media on which the encoding device 110 can directly transmit encoded video data to the decoding device 120 in real time. In this example, the encoding device 110 may modulate the encoded video data according to a communication standard and transmit the modulated video data to the decoding device 120. Here, the communication medium includes a wireless communication medium (e.g., a radio frequency spectrum), and optionally, the communication medium may further include a wired communication medium (e.g., one or more physical transmission lines).

[0021] In another example, channel 130 may include a storage medium that stores video data encoded by the encoding device 110. The storage medium may include a variety of local-access data storage media such as optical discs, DVDs, and flash memory. In this example, the decoding device 120 may obtain the encoded video data from the storage medium.

[0022] In another example, channel 130 may include a storage server that stores the video data encoded by the encoding device 110. In this example, the decoding device 120 may download the encoded video data stored from the storage server. Selectively, the storage server may store the encoded video data and transmit the encoded video data to the decoding device 120, for example, a web server (e.g., for a website), a File Transfer Protocol (FTP) server, etc.

[0023] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113, where the output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0024] In some embodiments, the encoding device 110 may further include a video source 111 in addition to the video encoder 112 and the input interface 113.

[0025] The video source 111 may include at least one of a video acquisition device (e.g., a video camera), a video archive, a video input interface, and a computer graphics system, where the video input interface is used to receive video data from a video content provider and the computer graphics system is used to generate video data.

[0026] The video encoder 112 encodes video data from the video source 111 and generates a bitstream. The video data may include one or more pictures or sequences of pictures. The bitstream contains encoding information for the pictures or sequences of pictures in bitstream format. The encoding information may include encoded image data and related data. The related data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntactic structures. The SPS may include parameters that apply to one or more sequences. The PPS may include parameters that apply to one or more pictures. A syntactic structure refers to a set of zero or more syntactic elements arranged in a specified order within the bitstream.

[0027] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may be further stored in a storage medium or storage server for later reading by the decoding device 120.

[0028] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122.

[0029] In some embodiments, the decoding device 120 may further include a display device 123 in addition to the input interface 121 and the video decoder 122.

[0030] Here, the input interface 121 may include a receiver and / or a modem. The input interface 121 may receive encoded video data via channel 130.

[0031] The video decoder 122 is used to decode the encoded video data, obtain the decoded video data, and transmit the decoded video data to the display device 123.

[0032] The display device 123 displays the decoded video data. The display device 123 may be integrated with the decoding device 120 or installed outside the decoding device 120. The display device 123 may include a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0033] Furthermore, Figure 1 is merely an example, and the technical solutions of the embodiments of this application are not limited to Figure 1. For example, the technology of this application may also be applied to one-sided video encoding or one-sided video decoding.

[0034] The following describes the video encoding framework according to the embodiment of the present application.

[0035] Figure 2 is a schematic block diagram of a video encoder according to an embodiment of the present invention. It is understood that the video encoder 200 may be used for lossy compression or lossless compression of an image. This lossless compression may be visually lossless compression or mathematically lossless compression.

[0036] The video encoder 200 may be applied to image data in luminance-chromaticity (YCbCr, YUV) format. For example, the YUV ratio may be 4:2:0, 4:2:2, or 4:4:4, where Y represents luminance (Luma), Cb (U) represents blue chromaticity, Cr (V) represents red chromaticity, and U and V represent chromaticity (Chroma) to describe color and saturation. For example, in the color format, 4:2:0 means that there are four luminance components and two chromaticity components (YYYYCbCr) for every four pixels, 4:2:2 means that there are four luminance components and four chromaticity components (YYYYCbCrCbCr) for every four pixels, and 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0037] For example, the video encoder 200 reads video data and divides each frame image in the video data into multiple coding tree units (CTUs). In some examples, CTUs are called "tree blocks," "largest coding units" (LCUs), or "coding tree blocks" (CTBs). Each CTU may be associated with a pixel block of the same size in the image. Each pixel may correspond to one luminance (or luma) sample and two chrominance (or chroma) samples. Therefore, each CTU may be associated with one luminance sample block and two chrominance sample blocks. The size of a single CTU may be, for example, 128×128, 64×64, or 32×32. A single CTU can be further divided into multiple coding units (CUs) for coding, and the CUs may be rectangular blocks or square blocks. The CU is further divided into a Prediction Unit (PU) and a Transform Unit (TU), separating encoding, prediction, and transformation to increase processing flexibility. In one example, the CTU is divided into CUs using a quadtree scheme, and the CU is then divided into TUs and PUs using a quadtree scheme.

[0038] Video encoders and video decoders can support a variety of PU sizes. Assuming a specific CU size of 2N×2N, video encoders and video decoders can support 2N×2N or N×N PU sizes for intra-prediction, and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-prediction. Video encoders and video decoders can further support asymmetric PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-prediction.

[0039] In some embodiments, as shown in Figure 2, the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / inverse quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. The video encoder 200 may further include many, fewer, or different functional components.

[0040] Selectively, in this application, the current block may be called the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block may also be called the prediction image block or image prediction block, and the reconstructed image block may also be called the reconstruction block or image reconstruction block.

[0041] In some embodiments, the prediction unit 210 includes an inter-prediction unit 211 and an intra-estimation unit 212. Because there is a strong correlation between adjacent pixels in one intra of video, the intra-prediction method is used in video coding and decoding techniques to remove spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent inters in video, the inter-prediction method is used in video coding and decoding techniques to remove temporal redundancy between adjacent inters, thereby improving coding efficiency.

[0042] The interprediction unit 211 may be used for interprediction, which may include motion estimation and motion compensation, and may refer to image information from different frames. Interprediction is used to find reference blocks from reference frames using motion information, generate prediction blocks based on the reference blocks, and remove temporal redundancy. The frames used for interprediction can be P frames and / or B frames, where P frames refer to forward prediction frames and B frames refer to bidirectional prediction frames. Interprediction finds reference blocks from reference frames using motion information and generates prediction blocks based on the reference blocks. Motion information includes a list of reference frames in which the reference frames are located, a reference frame index, and a motion vector. The motion vector may be an integer pixel or a fractional pixel. If the motion vector is a fractional pixel, it is necessary to generate the required fractional pixel blocks in the reference frame using an interpolation filter, and here the integer pixel or fractional pixel block of the reference inter found based on the motion vector is called the reference block. In some techniques, the reference block is directly used as the prediction block, and in some techniques, further processing is performed on the reference block to generate the prediction block. Generating a prediction block by further processing a reference block can also be understood as generating a new prediction block by first using the reference block as a prediction block and then further processing it based on that prediction block.

[0043] The intra-estimation unit 212 refers only to information from the same frame image, predicts pixel information within the current encoded image block, and is used to remove spatial redundancy. The frame used for intra-prediction may be an I-frame.

[0044] Intra-prediction has a variety of prediction modes. Taking the international digital video coding standard H-series as an example, the H.264 / AVC standard has 8 types of angle prediction modes and 1 type of non-angle prediction mode, while H.265 / HEVC is extended to 33 types of angle prediction modes and 2 types of non-angle prediction modes. The intra-prediction modes used in HEVC include Planar mode, DC, and 33 types of angle modes, for a total of 35 types of prediction modes. The intra-prediction modes used in VVC include Planar, DC, and 65 types of angle modes, for a total of 67 types of prediction modes.

[0045] Furthermore, with the increase in angle modes, intra-prediction becomes more accurate, better meeting the evolving needs of high-resolution and ultra-high-resolution digital video.

[0046] The residual unit 220 can generate residual blocks of the CU based on the pixel blocks of the CU and the prediction blocks of the CU's PU. For example, by generating residual blocks of the CU, the residual unit 220 ensures that each sample in the residual block has a value equal to the difference between the sample in the pixel block of the CU and the corresponding sample in the prediction block of the CU's PU.

[0047] The conversion / quantization unit 230 can quantize the conversion coefficients. The conversion / quantization unit 230 can quantize the conversion coefficients associated with the TU of the CU based on the quantization parameter (QP) value associated with the CU. The video encoder 200 can adjust the degree of quantization applied to the conversion coefficients associated with the CU by adjusting the QP value associated with the CU.

[0048] The inverse transform / inverse quantization unit 240 can apply inverse quantization and inverse transform to the quantized transformation coefficients, respectively, and reconstruct the residual block from the quantized transformation coefficients.

[0049] The reconstruction unit 250 can add the samples of the reconstructed residual blocks to the corresponding samples of one or more prediction blocks generated by the prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of the CU in this manner, the video encoder 200 can reconstruct the pixel blocks of the CU.

[0050] The loop filter unit 260 processes the pixels after inverse transformation and inverse quantization to compensate for distortion information and provide a good reference for subsequent encoded pixels. For example, deblocking filtering can be performed to reduce the blocking effect of pixel blocks associated with the CU.

[0051] In some embodiments, the loop filter unit 260 includes a deblocking filter unit and a sample adaptive compensation / adaptive loop filter (SAO / ALF) unit, where the deblocking filter unit is used to eliminate blocking effects and the SAO / ALF unit is used to eliminate ringing effects.

[0052] The decoded image buffer 270 can store the reconstructed pixel blocks. The interpretation unit 211 can perform interpretation on other PUs of other images using the reference image containing the reconstructed pixel blocks. In addition, the intraestimation unit 212 can perform intraprediction on other PUs in the same image as the CU using the reconstructed pixel blocks in the decoded image buffer 270.

[0053] The entropy coding unit 280 can receive quantized transformation coefficients from the transformation / quantization unit 230. The entropy coding unit 280 can generate entropy-coded data by performing one or more entropy coding operations on the quantized transformation coefficients.

[0054] Figure 3 is a schematic block diagram of a video decoder according to an embodiment of the present invention.

[0055] As shown in Figure 3, the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / inverse transform unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded image buffer 360. The video decoder 300 may further include many, fewer, or different functional components.

[0056] The video decoder 300 can receive a bitstream. The entropy decoding unit 310 can extract syntactic elements from the bitstream by analyzing it. As part of the bitstream analysis, the entropy decoding unit 310 can analyze the syntactic elements in the bitstream after entropy coding. The prediction unit 320, the inverse quantization / inverse transformation unit 330, the reconstruction unit 340, and the loop filter unit 350 can decode the video data based on the syntactic elements extracted from the bitstream, i.e., generate the decoded video data.

[0057] In some embodiments, the prediction unit 320 includes an intra-estimation unit 322 and an inter-prediction unit 321.

[0058] The intra-estimation unit 322 can generate prediction blocks for the PU by performing intra-prediction. The intra-estimation unit 322 can generate prediction blocks for the PU based on spatially adjacent pixel blocks of the PU using the intra-prediction mode. The intra-estimation unit 322 can further determine the intra-prediction mode for the PU based on one or more syntactic elements analyzed from the bitstream.

[0059] The interprediction unit 321 can construct a first reference image list (list 0) and a second reference image list (list 1) based on syntactic elements analyzed from the bitstream. Furthermore, if the PU uses interprediction coding, the entropy decoding unit 310 can analyze the motion information of the PU. Based on the motion information of the PU, the interprediction unit 321 can determine one or more reference blocks of the PU. Based on one or more reference blocks of the PU, the interprediction unit 321 can generate prediction blocks for the PU.

[0060] The inverse quantization / inverse transformation unit 330 can inverse quantize (i.e., perform inverse quantization processing) the transformation coefficients associated with the TU. The inverse quantization / inverse transformation unit 330 can determine the degree of quantization using the QP value associated with the CU of the TU.

[0061] After the inverse quantization of the conversion coefficients, the inverse quantization / inverse transformation unit 330 can generate residual blocks associated with the TU by applying one or more inverse transformations to the inversely quantized conversion coefficients.

[0062] The reconstruction unit 340 can reconstruct the pixel blocks of the CU using the residual blocks associated with the TU of the CU and the predicted blocks of the PU of the CU. For example, the reconstruction unit 340 can reconstruct the pixel blocks of the CU by adding the samples of the residual blocks to the corresponding samples of the predicted blocks and obtain a reconstructed image block.

[0063] The loop filter unit 350 can perform deblocking filtering to reduce the blocking effect of pixel blocks associated with the CU.

[0064] The video decoder 300 can store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent predictions, or it may transmit the reconstructed image to a display device for display.

[0065] The basic flow of video encoding and decoding is as follows: On the encoding side, a single frame image is divided into blocks, and for the current block, the prediction unit 210 generates a predicted block of the current block using intra-prediction or inter-prediction. The residual unit 220 can calculate a residual block, i.e., the difference between the predicted block and the original block of the current block, based on the predicted block and the original block of the current block, and this residual block is also called residual information. This residual block undergoes processes such as transformation and quantization by the transformation / quantization unit 230 to remove information insensitive to the human eye and eliminate visual redundancy. Selectively, the residual block before transformation and quantization by the transformation / quantization unit 230 is called a time-domain residual block, and the time-domain residual block after transformation and quantization by the transformation / quantization unit 230 is called a frequency-domain residual block or frequency-range residual block. The entropy encoding unit 280 receives the quantized transformation coefficients output from the transformation / quantization unit 230, performs entropy encoding on these quantized transformation coefficients, and outputs a bitstream. For example, the entropy coding unit 280 can remove character redundancy based on the target context model and the probabilistic information of the binary bitstream.

[0066] On the decoding side, the entropy decoding unit 310 analyzes the bitstream to obtain prediction information and quantization coefficient matrix for the current block. The prediction unit 320 generates a predicted block for the current block using intra-prediction or inter-prediction based on the prediction information. The inverse quantization / inverse transform unit 330 uses the quantization coefficient matrix obtained from the bitstream to perform inverse quantization and inverse transform on the quantization coefficient matrix to obtain the residual block. The reconstruction unit 340 adds the predicted block and the residual block to obtain the reconstructed block. The reconstructed block constitutes a reconstructed image, and the loop filter unit 350 performs loop filtering on the reconstructed image based on the image or the block to obtain the decoded image. The encoding side also needs to perform the same operations as the decoding side to obtain the decoded image. This decoded image is also called the reconstructed image, and the reconstructed image can be used as a reference frame for inter-prediction of subsequent frames.

[0067] Furthermore, the block partitioning information determined by the encoding side, as well as mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, are included in the bitstream as needed. The decoding side analyzes the bitstream and, based on the existing information, determines the same block partitioning information as the encoding side, as well as mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding side matches the decoded image obtained by the decoding side.

[0068] The above describes the basic flow of a video codec under a block-based mixed coding framework. As technology advances, some modules or steps of this framework or flow may be optimized. This application applies to, but is not limited to, the basic flow of a video codec under the said block-based mixed coding framework.

[0069] In some embodiments, the current block may be the current coding unit (CU) or the current prediction unit (PU), etc. Due to the need for parallel processing, the image may be divided into slices, etc., and slices within the same image can be processed in parallel, i.e., there is no data dependency between them. "Frame" is a general term and can usually be understood as one image. The frames described in this application may be replaced with images or slices, etc.

[0070] As can be seen from the above, interpretation eliminates redundancy by utilizing temporal correlation. To prevent flickering from being visible to the human eye, typical video frame rates are 30 frames / second, 50 frames / second, 60 frames / second, and even 120 frames / second. In such videos, there is a high correlation between adjacent frames in the same scene, and interpretation technology utilizes this correlation to predict what to encode now by referring to the content of already encoded and decoded frames. Interpretation can significantly improve encoding performance.

[0071] The most basic interpretation method is translational prediction. Translational prediction assumes that the content being predicted is in translational motion between the current image and the reference image. For example, if the content of the current block (encoded unit or prediction unit) is in translational motion between the current image and the reference image, this content can be found in the reference image via the motion vector (MV) and used as the prediction block for the current block. Translational motion accounts for a large portion of video, and stationary backgrounds, objects that are translated as a whole, and lens translation can all be handled by translational prediction.

[0072] Natural video contains content that is not simply translation. For example, there are subtle changes in shape and color during the translation process. Bidirectional prediction finds two reference blocks from a reference image and weights-averages these two reference blocks to obtain a predicted block that is as similar as possible to the current block. For example, in a given scene, finding reference blocks from before and after the current frame and weighting-averaging them may result in a more similar predicted block to the current block than a single reference block could. Based on this, bidirectional prediction further improves compression performance based on unidirectional prediction.

[0073] The picture order count (POC) can be used as an identifier for an image. In a video sequence, each image has a unique POC, and in the embodiments of this application, the order of the POCs and the playback order are considered to be the same. A P image (P Frame) is an image that can be predicted using only reference images prior to the current image. The current reference image has only one reference image list, denoted as RPL0. Here, RPL can be understood as an abbreviation for Reference Picture List. All reference images in the reference image list RPL0 are reference images prior to the current image. A B image (B Frame) was previously an image that could be predicted using both reference images prior to the current image and reference images later than the current image. A B image has two reference image lists, denoted as RPL0 and RPL1. One arrangement is that RPL0 contains all reference images prior to the current image, and RPL1 contains all reference images later than the current image. For the current block, we may refer only to the reference block of a specific image in RPL0, which is also called forward prediction; we may refer only to the reference block of a specific image in RPL1, which is also called backward prediction; and we may refer to both the reference block of a specific image in RPL0 and the reference block of a specific image in RPL1 simultaneously, which is also called bidirectional prediction. A simple way to refer to two reference blocks simultaneously is to obtain the predicted block of the current block by averaging the pixels at each corresponding position in the two reference blocks. Later, the B image was no longer limited to the case where RPL0 contains a reference image where the POC is earlier than the current image, or where RPL1 contains a reference image where the POC is later than the current image. Therefore, RPL0 may contain a reference image where the POC is later than the current image, and RPL1 may contain a reference image where the POC is earlier than the current image. The current block can simultaneously refer to a reference image where the POC is earlier than the current image, or simultaneously refer to a reference image where the POC is later than the current image. Such a B image is also called a generalized B image.

[0074] In a Random Access (RA) configuration, the coding order and the Point of Control (POC) order are different. Thus, because the B image can reference information prior to and after the current image, coding performance is significantly improved. Figure 4 shows the classic GOP (group of pictures) structure of RA. The arrows in the figure represent reference relationships. The I image does not require a reference image; after the I image with POC 0 is decoded, the P image with POC 4 is decoded. When decoding the P image with POC 4, the I image with POC 0 can be referenced. Next, the B image with POC 2 is decoded. When decoding the B image with POC 2, the I image with POC 0 and the P image with POC 4 can be referenced, and so on.

[0075] In the LD (Low Delay) configuration, the coding order and POC order are the same. Therefore, the current image can only refer to information prior to the current image. The Low Delay configuration can be further divided into Low Delay P and Low Delay B. Low Delay P is the conventional Low Delay configuration. Its typical structure is IPPP..., meaning that first one I image is coded and decoded, and all subsequent images are P images. The typical structure of Low Delay B is IBBB..., and the difference from Low Delay P is that each inter-image is a B image, meaning that two reference image lists are used, and the current block can simultaneously refer to the reference block of a specific image in RPL0 and the reference block of a specific image in RPL1.

[0076] Generally, the compression efficiency of the RA configuration is higher than that of the LD configuration, and the compression efficiency of the LDB configuration is higher than that of the LDP configuration. This is because, on the one hand, bidirectional predictions can refer to information in the reverse direction, and on the other hand, prediction errors can be reduced by techniques such as weighted averaging.

[0077] A single reference image list for a current image can contain up to several reference images (e.g., two, three, or four). When encoding a current image, which reference images are included in RPL0 and RPL1 respectively is determined by a certain arrangement or algorithm and is not the focus of this application. However, the same reference image can appear simultaneously in both RPL0 and RPL1. That is, the codec allows a current block to simultaneously reference two reference blocks of the same reference image.

[0078] A codec typically uses an index value within a list of reference images to map to a reference image. If the list of reference images is 4 long, then there are four possible values ​​for index: 0, 1, 2, and 3. For example, suppose the current frame's RPL0 has four reference images with POC values ​​of 5, 4, 3, and 0. In this case, index 0 in RPL0 corresponds to the reference image with POC 5, index 1 corresponds to the reference image with POC 4, index 2 corresponds to the reference image with POC 3, and index 3 corresponds to the reference image with POC 0.

[0079] Interpretation uses motion information to represent "motion." Basic motion information includes information about a reference picture and motion vectors (MV). For a block to be able to use bidirectional prediction, it naturally needs to find two reference blocks, which requires information about two sets of reference pictures and motion vectors. Each of these sets can be understood as unidirectional motion information, and these two sets can be combined to form bidirectional motion information. In concrete implementation, unidirectional and bidirectional motion information can use the same data structure, where both sets of reference picture information and motion vector information are valid in bidirectional motion information, and where one set of reference picture information and motion vector information is invalid in unidirectional motion information. Here, "valid" can also be said as "used," and "invalid" can also be said as "not used."

[0080] VVC supports two reference image lists, denoted as RPL0 and RPL1. For the bidirectional motion information described above, VVC uses the reference image index refIdxL0 and motion vector mvL0 corresponding to RPL0, the reference image index refIdxL1 and motion vector mvL0 corresponding to RPL1, and the reference image index refIdxL1 and motion vector mvL0 corresponding to RPL1. The reference image index corresponding to RPL0 and the reference image index corresponding to RPL1 here can be understood as the reference image information described above. VVC uses two flags, denoted as predFlagL0 and predFlagL1, to indicate whether to use the motion information corresponding to RPL0 and the motion information corresponding to RPL1, respectively. predFlagL0 and predFlagL1 can also be understood as indicating whether the unidirectional motion information described above is "valid." Therefore, although a data structure called motion information is not explicitly mentioned in VVC, motion information is represented using the reference image index, motion vector, and "valid" flag bit corresponding to each reference image list together. In the VVC standard text, motion information does not appear; instead, motion vectors are used, and the reference image index and the flag indicating whether to use the corresponding motion information can be considered appendages of the motion vectors. In this application, we still use "motion information" for ease of explanation, but it should be understood that we can also explain using "motion vectors." "Motion information" can also be called "motion parameters."

[0081] For a two-dimensional image, the motion vector may be represented as (x,y), that is, the horizontal and vertical components. Since video is always represented by pixels, there is distance between pixels, and the movement of an object does not always correspond to an integer pixel distance between adjacent images. For example, in a video of a distant scene, if the distance between two pixels for a distant object is 1 meter, and this object moves 0.5 meters in time between two frames, then this scene cannot be adequately represented by an integer pixel motion vector. Therefore, motion vectors can be performed down to fractional pixel levels such as 1 / 2 pixel precision, 1 / 4 pixel precision, 1 / 8 pixel precision, and 1 / 16 pixel precision, allowing for a finer representation of movement. The pixel value at the fractional pixel position in the reference image can then be obtained using an interpolation method.

[0082] In the translational prediction described above, both unidirectional and bidirectional predictions are block-based, for example, coding units or prediction units. That is, predictions are made on a pixel matrix basis. The most basic blocks are rectangular blocks, such as squares and rectangles. In video coding standards such as HEVC and VVC, the encoder can determine the size and division method of coding and prediction units based on the video content. Areas with simple textures or motions tend to use larger blocks, while areas with complex textures or motions tend to use smaller blocks. The deeper the hierarchy of block divisions, the more complex and closer to the actual texture or motion can be divided into blocks, but the overhead for characterizing these divisions also increases accordingly. Motion information may also need to be transmitted in the bitstream. And generally, the finer the block divisions, the greater the overhead for motion information.

[0083] The most primitive way to represent motion information is to directly write the complete motion information. Later, experts discovered that motion vectors can be represented using motion vector prediction (MVP) and motion vector difference (MVD). That is, MV = MVP + MVD. If the MVP is more accurate, the MVD will be smaller than MVP, and the overhead occupied by the bitstream will also be smaller than MVP.

[0084] It is understood that each inter-coded block requires motion information. To simplify the problem, we assume that CU partitioning is equal to PU partitioning and TU partitioning; that is, each coded unit has one predictive unit of the same size and position, and one transform unit of the same size and position. In practice, as CU partitioning becomes more flexible, VVC tends to weaken PU and TU compared to HEVC. Differences in any of the rings of predictive, transform, quantization, or entropy coding can cause CU partitioning. For example, if the motion information of two regions is different, the encoder may partition these two regions into different CUs. Also, if the motion information of two regions is the same or similar, but the residual characteristics are significantly different, the encoder may partition these two regions into different CUs. How to partition is determined based on the overall compression efficiency and does not necessarily depend entirely on a single factor. Therefore, the same object, i.e., regions with the same or similar motion, may be partitioned into different CUs.

[0085] As an example, Figure 5 shows an example of HEVC. Figure a is the original figure, in which there is a horizontal bar moving in the direction indicated by the arrow, and the movement of the background area is small. Figure b shows the block division of HEVC, and figure c is figure b with the boundaries of blocks with the same motion information removed. It can be seen that many adjacent blocks use the same motion information. In this situation, encoding motion information individually for each block would result in obvious redundancy. As mentioned above, the complete motion information of VVC includes the reference image index of RPL0, a flag indicating whether MV is used, the reference image index of RPL1, and a flag indicating whether MV is used. The basic principle of merge mode is that the current block can inherit the motion information of adjacent blocks, i.e., the reference image information and motion vector information.

[0086] Merge mode allows the construction of a single merge candidate list. When the current block uses Merge mode, a single index is used to indicate which motion information the current block should merge, thereby eliminating the need to encode the complete motion information. When constructing the merge candidate list, motion information of adjacent blocks in the current block's spatial domain, motion information in the temporal domain, motion information of blocks that are not adjacent in the spatial domain, motion information of blocks that are not adjacent in the temporal domain, motion information based on history, and synthesized motion information can be added.

[0087] In the spatial domain, adjacent blocks refer to blocks adjacent to the current block within the same image, while non-adjacent blocks refer to blocks not adjacent to the current block within the same image. The motion information in the time domain and the motion information of non-adjacent blocks in the time domain refer to the motion information of the specified position in the collocated reference image. Exemplarily, as shown in Figure 6, the large gray block is the current block, where positions 1, 2, 3, 4, and 5 are the positions of adjacent blocks in the spatial domain used by Merge, and the other dark gray positions are the positions of non-adjacent blocks in the spatial domain used by Merge. Position 6 is the position where motion information in the time domain is used, and if the corresponding position at the lower right corner of the current block is unavailable, the corresponding position at the center of the current block is used. The other light gray positions are the positions where motion information of non-adjacent blocks in the time domain is used. Time domain motion information is derived based on the motion information at the corresponding position in the collocated reference image.

[0088] Time-domain motion information prediction is used as a complement to spatial-domain motion information prediction. Generally, correlations between adjacent regions in the same image are stronger than correlations between different images. However, there are situations where time-domain motion information is more useful. To give a simple example, if the current block and its surrounding adjacent blocks in the current image belong to different objects and have completely different movements, then the movement of blocks belonging to the same object as the current block in a reference image can provide better motion information prediction for the current block.

[0089] As an example, as shown in Figure 7, the motion vector of an isotopic block in an isotopic reference image (here, the block from which time-domain motion information is acquired is called an isotopic block) is the vector from its isotopic reference image col_pic to the reference image col_ref of that isotopic block. For the current block, the required motion vector is the vector from the current image curr_pic to the reference image curr_ref of the current block. Let td be the POC distance between col_pic and col_ref, and tb be the POC distance between curr_pic and curr_ref. Assuming that the motion in the current block is invariant from the motion in the isotopic block, the scaling ratio may be determined based on td and tb. If the motion vector of the isotopic block is (col_mv_x, col_mv_y), the time-domain motion vector prediction (tmvp_x, tmvp_y) can be derived as shown in equation (1) below.

number

[0090] In VVC, the smallest unit of motion information stored in the co-reference image is 4x4. That is, one set of motion information is stored in each 4x4 subblock. If hardware implementation costs are not considered, it is understood that the co-reference image can also store one set of motion information for each pixel.

[0091] VVC introduces subblock-based temporal motion vector prediction, or SbTMVP (Subblock-based temporal motion vector prediction). Typically, MVP (motion vector prediction) and TMVP (temporal motion vector prediction) apply to the entire block, meaning the entire block shares the same MVP. SbTMVP, on the other hand, is subblock-based, allowing it to obtain one MVP for each subblock. This is the essential difference between SbTMVP and TMVP.

[0092] On the other hand, TMVP currently positions an isotopic block using the position of the lower right corner of the block or the position of the center of the block, while SbTMVP determines its position by finding a motion offset based on the movement of the surrounding blocks. In VVC, if the block at position A1 references an isotopic reference image, its motion displacement is set to the motion vector that A1 uses for the isotopic reference image. Otherwise, the motion displacement is set to (0,0). As shown in Figure 8, the position is found according to the motion displacement, and then the MV of the position corresponding to each subblock of the “isotopic block” is scaled to obtain the MVP of each subblock.

[0093] Merge mode uses motion information selected from the merge candidate list directly as motion information for the current block. In actual video, there are sometimes some differences between the actual motion vector of the current block and the motion vector in the selected merge candidate list. Merge mode with MVD (MMVD) is a special merge mode in VVC that uses a method to efficiently encode MVD in such cases. Normal merge does not require encoding / decoding of MVD (motion vector difference). Normal inter mode requires direct encoding / decoding of MVD. As shown in Figure 9, MMVD utilizes the characteristic that MVD is often distributed in a single horizontal or single vertical direction, with many MVDs for small values ​​and fewer MVDs as the value increases.

[0094] As shown in Figure 9, MMVD can only represent the MVD of a specific number in a specific direction, and cannot represent an arbitrary MVD. The direction of the MVD is represented using mmvd_direction_idx. Of course, it can also be understood as whether the x and y of the MVD are non-zero and have a positive or negative sign. The magnitude MmvdDistance is represented by mmvd_distx, which is the magnitude of the absolute value of the non-zero x and y of the MVD.

[0095] Table 1 shows, as an example, the relationship between mmvd_distance_idx[x0][y0] and MmvdDistance[x0][y0].

[0096] [Table 1]

[0097] Here, ph_mmvd_fullpel_only_flag is a picture header flag, and it can be set to one of two different combinations of MMVDs.

[0098] Table 2 shows an example of the relationship between mmvd_direction_idx[x0][y0] and MmvdSign[x0][y0].

[0099] [Table 2]

[0100] In some embodiments, the MVD of the MMVD is obtained according to the following formula (2).

number

[0101] The motion described above is the simplest and most common translational motion. In the real world, motion is not limited to translation; there are also shrinking, expanding, rotating, perspective motion (objects closer to the lens appear larger, and objects further away appear smaller), and many other irregular forms of motion. Affine can be used to represent motion that is more complex than translation. As shown in Figure 10, Affine uses a linear model to calculate the motion vector of each subblock or each pixel within the current block, based on the motion vectors of two control points (four parameters; one motion vector contains two parameters, x and y) or three control points (six parameters).

[0102] For example, for a 4-parameter Affine model, the motion vector at the current (x,y) position within the block is derived according to equation (3) below.

number

[0103] For example, for a 6-parameter Affine model, the motion vector at the current (x,y) position within the block is derived according to equation (4) below.

number

[0104] However, (mv0x, mv0y) is the motion vector of the control point at the top left corner of the block, (mv1x, mv1y) is the motion vector of the control point at the top right corner of the block, and (mv2x, mv2y) is the motion vector of the control point at the bottom left corner of the block.

[0105] In VVC, Affine simplifies the complexity of hardware implementation by dividing the current block into 4x4 sub-blocks and calculating one motion vector (MV) for each sub-block to perform motion compensation. Exemplary, Figure 10B shows an example of Affine deriving motion vectors on a sub-block basis. As hardware processing power improves, Affine is expected to become capable of pixel-based processing as well. That is, one motion vector will be derived for each pixel, and motion compensation will be performed for that single pixel based on that motion vector.

[0106] Affine can derive motion vectors for each subblock or each pixel using only a few control points, enabling more precise predictions compared to motion compensation based on the entire block. On the other hand, Affine's overhead is much smaller compared to partitioning into finer CUs.

[0107] In some embodiments, HEVC supports up to 64x64 CTUs and may perform recursive quadtree partitioning. VVC supports a more flexible set of block partitioning methods than HEVC, supporting up to 128x128 CTUs and including quadtree, ternary, and binary tree partitioning. While these partitioning methods offer increasing flexibility in block partitioning, they always partition into rectangular blocks, regardless of whether it is CU, PU, ​​or TU. Note that PU and TU partitioning are less pronounced in VVC. Texture and motion boundaries in natural images are diverse. For example, when encountering the boundary of a diagonal object, simply attempting to approximate the boundary using rectangular blocks will result in partitioning into many small blocks, significantly increasing overhead. Geometric partitioning mode (GPM) can better handle textures and boundaries in natural images.

[0108] GPM uses two prediction blocks of the same size as the current block. In the GPM prediction blocks, some pixel locations use 100% of the pixel values ​​from the corresponding locations in the first prediction block, and other pixel locations use 100% of the pixel values ​​from the corresponding locations in the second prediction block, and in boundary or transition regions, the pixel values ​​from the corresponding locations in these two prediction blocks are used in a constant ratio. The weights in boundary regions also transition gradually. Of course, transition regions do not have to be used to accommodate scenarios such as screen content encoding. Specifically, how these weights are assigned is determined by the GPM's "splitting" mode. The weight of each pixel location is determined according to the GPM's "splitting" mode. Of course, under certain circumstances, such as when the block size is very small, certain GPM modes may not guarantee that some pixel locations will use 100% of the pixel values ​​from the first prediction block and other pixel locations will use 100% of the pixel values ​​from the second prediction block. It can also be thought of as using two prediction blocks of different sizes than the current block, i.e., taking the necessary parts from each. Remove the parts with a weight of 0. This is a practical issue and not the focus of this application.

[0109] Exemplary, Figure 11 shows the weight diagrams of 64 different modes on a square block of GPM in VVC. Black indicates that the weight value of the corresponding position in the first prediction block is 0%, white indicates that the weight value of the corresponding position in the first prediction block is 100%, and gray areas indicate, by shade, that the weight value of the corresponding position in the first prediction block is greater than 0% but less than 100%. The weight value of the corresponding position in the second reference block is 100% minus the weight value of the corresponding position in the first reference block.

[0110] GPM can be described as a prediction mode or prediction method because it ultimately generates prediction blocks. Furthermore, GPM can be described as a "splitting" mode, simulating splitting of prediction blocks and achieving PU splitting, but without actually performing any substantial splitting. The first and second prediction blocks used in the aforementioned GPM may be prediction blocks generated by intra-prediction, prediction blocks generated by unidirectional prediction between frames, or prediction blocks generated by bidirectional prediction between frames.

[0111] In some implementations, because consumer video bitrates are limited, video compression typically requires a trade-off between bitstream overhead and distortion. Taking block division as an example, for the same content, within a certain range, finer divisions result in higher overhead and lower distortion, while coarser divisions result in lower overhead and higher distortion. Similarly, with motion encoding, within a certain range, for the same content, more accurate motion information results in higher overhead and lower distortion, while coarser motion information results in lower overhead and higher distortion. Some decoding methods utilize the decoding information for processing and calculations without occupying overhead, achieving improvements in motion information, enhanced prediction, and reduced distortion. Not occupying overhead means that the encoder processes automatically based on the available information, rather than giving instructions based on the original image. Two typical decoding methods in VVC are DMVR (Decoder-side motion vector refinement) and BDOF (bi-directional optical flow).

[0112] One condition for DMVR to be activated in VVC is that the two reference images in the current block are located before and after the current image, respectively, and the distances between the two reference images and the current image are equal. Another activation condition is that the current CU is using a whole-block merge mode (including skip). The whole-block refers to a sub-block-based merge, such as SbTMVP or affine merge, because merge modes can easily lead to inaccurate motion vectors. Other conditions will not be repeated here. DMVR in VVC utilizes bilateral matching (BM), that is, calculating the matching cost of the reference blocks on both sides, e.g., SAD (sum of absolute difference). DMVR searches for the matching cost of the MVs around the original MV, and during movement, the MVs of the two reference images move mirror-image. That is, based on each original MV, one moves by MVdiff and the other by -MVdiff (see Figure 12). Since fractional pixel searches are also supported during the search, DMVR may find MVs with higher accuracy than the original MVs. The search is performed according to certain rules, generally starting with integer pixel MVs within a certain range to find the integer pixel MV with the smallest matching cost, and then searching for fractional pixel MVs based on that integer pixel MV. If an MV with a smaller matching cost than the original MV is found, motion compensation prediction is performed using the MV with the smaller matching cost. Theoretically, MVs improved by DMVR can be used for MV preservation or the use of surrounding blocks. For example, when the current block is building a merge candidate list, if the surrounding blocks have improved their MVs with DMVR, using the improved MVs to build the merge candidate list will result in a better compression effect. However, due to hardware implementation considerations, this is not done in VVC.

[0113] DMVR is capable of subblock-based processing. In fact, in VVC, if the horizontal or vertical size of a block exceeds 16 pixels, it is divided into subblocks of 16 pixels in size. This is, on the one hand, a consideration based on the complexity of hardware implementation, because DMVR needs to perform a search on the decoding side, and limiting the size of the subblocks can reduce the cache cost. On the other hand, dividing and processing into subblocks provides better flexibility, and since each subblock can independently improve the MV, it has the effect of improving the division accuracy to some extent, which also improves compression efficiency.

[0114] Bidirectional optical flow (BDOF) is also a typical decoding method. BDOF improves MV and prediction based on the principle of optical flow. Optical flow is the instantaneous velocity at which a pixel moves on the observation plane as an object moves in space. Optical flow has several basic assumptions. For example, brightness is invariant, meaning that the brightness does not change when the same target moves between different images. The motion is continuous or small, meaning that the target position does not change dramatically with time.

[0115] JPEG2026522707000008.jpg49169

[0116] For example, the process of deriving the predicted value includes the following:

[0117] JPEG2026522707000009.jpg19169

[0118] JPEG2026522707000010.jpg11168

number

[0119] JPEG2026522707000013.jpg7167

[0120] JPEG2026522707000014.jpg9167

number

[0121] JPEG2026522707000017.jpg8167

number

[0122] Then, each predicted value within this 4x4 sub-block is adjusted based on the motion vector deviation and gradient.

[0123] For example, the adjustment value for the predicted value is determined by the following equation (8).

number

[0124] Finally, the predicted value for the current block is adjusted based on the adjusted predicted value mentioned above to obtain the predicted BDOF value.

[0125] For example, the predicted value of BDOF is determined by the following equation (9).

number

[0126] BDOF's motion vector deviation can achieve very high accuracy, thereby making predictions more precise. Furthermore, its sub-block-based processing offers increased flexibility, similar to DMVR in these two respects.

[0127] Both DMVR and BDOF have the effect of improving motion vectors. DMVR is based on block matching, while BDOF is based on the principle of optical flow. These can be used in combination. An example is shown below. This may also be called Multi-pass decoder-side motion vector refinement (MDMVR).

[0128] For example, MDMVR may include: a first step of improving motion vectors by bidirectional matching based on the entire block; a second step of improving motion vectors by bidirectional matching based on subblocks, the subblock size of which may be 16x16; and a third step of improving motion vectors by bidirectional optical flow based on subblocks, the subblock size of which may be 8x8. Currently, it is possible to further enrich the number of steps. For example, a fourth step could be added, which involves improving motion vectors by bidirectional optical flow based on 4x4 subblocks. Alternatively, a step could be added that improves motion vectors by bidirectional optical flow based on points.

[0129] The template matching method was originally used in inter-block prediction, utilizing the correlation between adjacent pixels and using several regions around the current block as a template. When the current block is encoded and decoded, the left and upper sides have already been encoded and decoded according to the encoding order. Of course, in existing hardware decoder implementations, it is not guaranteed that the left and upper sides will be decoded when decoding of the current block begins. What is being discussed here is inter-block. For example, in HEVC, when an inter-encoded block generates a prediction block, it does not require the surrounding reconstructed pixels, so the inter-block prediction process can be performed in parallel. However, an intra-encoded block always requires the left and upper reconstructed pixels as reference pixels. Theoretically, the left and upper sides are obtainable. In other words, it is possible to achieve this with adjustments according to the hardware design. Relatively speaking, the right and lower sides are not obtainable under current standard encoding orders such as VVC.

[0130] As an example, as shown in Figure 13, the rectangular areas to the left and above the current block are set as templates. The height of the left template portion is usually the same as the height of the current block, and the width of the upper template portion is usually the same as the width of the current block. Of course, they may be different. The best matching position of the template is searched within the reference image, thereby determining the motion information of the current block, i.e., the motion vector. This process can be broadly described as searching within a certain range around a starting position in a given reference image. Search rules, such as the search range and search step length, may be set in advance. Each time the position is moved, the degree of matching between the template corresponding to that position and the template around the current block is calculated. This degree of matching can be measured by several distortion costs, such as SAD (sum of absolute difference), SATD (sum of absolute transformed difference, the transformation generally used in SATD is the Hadamard transform), and MSE (mean-square error). Smaller values ​​for SAD, SATD, MSE, etc., indicate a higher degree of matching. The cost is calculated using the predicted block of the template corresponding to that position and the reconstructed block of the template around the current block. In addition to searching for integer pixel positions, fractional pixel positions may also be searched, and the motion information of the current block is determined based on the position with the highest degree of matching found. By utilizing the correlation between adjacent pixels, motion information suitable for the template may also be suitable for the current block. Of course, the template matching method is not applicable to all blocks, so several methods can be used to determine whether the current block uses the template matching method described above. For example, a control switch can be used in the current block to indicate whether the template matching method is used. One name for this template matching method is DMVD (decoder side motion vector derivation).Both the encoder and decoder can use templates to perform searches and derive motion information, or find better motion information based on the original motion information. Furthermore, there is no need to transmit specific motion vectors or motion vector differences; the consistency of encoding and decoding is ensured because the encoder and decoder perform searches using the same rules. While template matching methods can improve compression performance, they introduce a certain degree of complexity on the decoding side because the decoding side also needs to perform a "search."

[0131] As can be seen from the above, there are various prediction methods. The encoder can determine which prediction mode, i.e., model, the current block will use, for example, whether to use merge mode or MMVD mode, and whether to predict the entire block or subblocks. When predicting the entire block, the merge candidate list still includes spatial domain motion vector predictions and time domain motion vector predictions. When predicting subblocks, the subblock candidates still include modes such as SbTMVP and Affine, and whether to use GPM mode. On the other hand, if more detailed and accurate information is conveyed to the decoder through the bitstream, the decoder can make better predictions, but the overhead increases accordingly. The encoder needs to make a trade-off between bitrate and distortion. The aforementioned methods such as merge, MMVD, GPM, SbTMVP, and Affine convey as much information as possible to the decoder in a more efficient way, and the decoder executes according to the instructions of the encoding end. On the other hand, decoding algorithms such as DMVR and BDOF compensate for distortion caused by inaccurate motion vectors through block matching or optical flow methods. This gives the encoder more room to pass less information to save bitstream overhead. The "smarter" the decoder, the less distortion it can perform if the encoding end gives the same instructions.

[0132] In encoding and decoding frameworks built with these existing technologies, the decoder obtains motion information instructions from the bitstream to obtain initial motion information, finds a reference block or region around the reference block based on the initial motion information, and improves the initial motion information and / or predicted values ​​based on the pixel value information of the reference block or region around the reference block. For example, DMVR searches around the initial MV, and the search process and the final MV selection depend on the block matching result. Similarly, BDOF also calculates information such as gradients based on pixel values ​​and calculates the instantaneous motion vector difference through the optical flow method.

[0133] However, currently, when the decoding side improves upon the initial motion information determined by decoding, the improvement effect is insufficient, and the predicted value of the current block determined by decoding is inaccurate, which affects the video encoding and decoding performance.

[0134] To solve the above technical problems, the present invention determines first motion information for the current block, then improves the first motion information based on the motion information of a reference image of the current block to obtain second motion information. Here, the motion information of the reference image is used to improve the first motion information and / or for block division. That is, in the embodiments of the present invention, when improving the first motion information, the motion information of the reference image is taken into consideration, and the first motion information is effectively improved to obtain accurate second motion information. As a result, when determining the predicted value of the current block based on this accurate second motion information, the prediction accuracy of the current block can be increased, and the video decoding performance can be further improved.

[0135] The video decoding method according to the embodiment of this application will be described below, with reference to Figure 14, using the decoding side as an example.

[0136] Figure 14 is a diagram showing a flow chart of a video decoding method according to one embodiment of the present invention. The embodiment of the present invention is applied to the video decoder shown in Figures 1 and 3. As shown in Figure 14, the method of the embodiment of the present invention includes the following:

[0137] S101 determines the first movement information for the current block.

[0138] The decoding method according to the embodiment of this application is applied to interpretation and is used to improve the motion information of the current block.

[0139] As mentioned above, due to bitrate considerations, the encoding end is increasingly carrying less prediction-related information to the bitstream. This means that the motion information of the current block, which the decoding side obtains based on the prediction-related information carried to the bitstream, is not sufficiently accurate. Therefore, the decoding side can improve the motion information determined by this decoding to enhance the prediction effect. However, the current motion information improvement process does not take into account the motion information of the reference image, and as a result, the improvement of the motion information is insufficient.

[0140] In the embodiments of this invention, when improving the motion information of the current block, the motion information of the reference image is taken into consideration to effectively improve the motion information of the current block, further increasing the prediction accuracy of the current block and improving the video decoding performance.

[0141] In some embodiments, it is understood that the first motion information of the current block described above is the initial motion information of the current block, i.e., the decoding side decodes the bitstream and obtains prediction-related information carried to the bitstream, and the motion information is determined based on said prediction-related information. This prediction-related information may include information such as the prediction mode.

[0142] In some embodiments, the first motion information of the current block described above may be understood as motion information after one or more improvements have already been made to the initial motion information of the current block, and this first motion information is further improved based on the motion information of the reference image by the method of the embodiments of the present application.

[0143] As mentioned above, bidirectional prediction uses motion information to represent "motion." Basic motion information includes information from a reference picture and information from motion vectors (MV). In some embodiments, when a block uses bidirectional prediction, it is necessary to find two reference blocks, which requires information from two sets of reference pictures and motion vectors. Each set can be understood as unidirectional motion information, and these two sets can be combined to form bidirectional motion information.

[0144] In some embodiments, the motion information of the embodiments of the present invention may refer to unidirectional motion information, i.e., information of a set of reference images and information of motion vectors.

[0145] In some embodiments, the motion information of the embodiments of the present invention may refer to bidirectional motion information, i.e., information from two sets of reference images and information from motion vectors.

[0146] In some embodiments, the motion information of the embodiments of the present invention may refer to multidirectional motion information, i.e., information including information from multiple sets of reference images and motion vector information.

[0147] In some embodiments, motion information may be represented using a reference image index, motion vector, and a "valid" flag bit corresponding to each reference image list.

[0148] The embodiments of this invention do not limit the specific method by which the decoding side determines the first movement information of the current block.

[0149] In one possible implementation, the encoding end includes the predicted mode of the current block in the bitstream. The decoding end then decodes the bitstream to obtain the predicted mode of the current block and, based on that predicted mode, determines the first motion information of the current block.

[0150] For example, the decryption side obtains the initial movement information of the current block based on its prediction mode and determines that initial movement information to be the first movement information.

[0151] In another example, the decryption side obtains the initial motion information of the current block based on its prediction mode, then improves that initial motion information, and determines the improved initial motion information as the first motion information. Exemplaryly, the decryption side may employ the DMVR and / or BDOF methods described above to improve the initial motion information. For example, the decryption side uses the DMVR improvement method to improve the initial motion information of the current block and obtain the first motion information. In another example, the decryption side uses the BDOF improvement method to improve the initial motion information of the current block and obtain the first motion information. In yet another example, the decryption side first improves the initial motion information using the DMVR improvement method, then further improves it using the BDOF improvement method, and obtains the first motion information. In yet another example, the decryption side first improves the initial motion information using the BDOF improvement method, then further improves it using the BDOF improvement method, and obtains the first motion information. Specific methods for improving DMVR and BDOF should be referred to in the above-described embodiments and will not be repeated here.

[0152] S102: Based on the motion information of the current block's reference image, the first motion information is improved and the second motion information of the current block is obtained.

[0153] Here, the motion information of the reference image is used to improve the first motion information and / or for block division.

[0154] As can be seen from the above, the motion information of the embodiment of the present invention includes information on a reference image and information on motion vectors. Based on this, the decoding side may obtain information on the reference image of the current block, for example, the index of the reference image, from the first motion information of the current block determined above, and further obtain the reference image of the current block from the reference image list based on that index.

[0155] In some embodiments, when the prediction of the current block is a unidirectional prediction in the embodiments of the present application, the current block corresponds to a list of reference images, which is denoted as RPL0. Next, the reference image of the current block is determined to have an index in this list of reference images RPL0, and based on that index, the reference image in the list of reference images RPL0 corresponding to that index is determined to be the reference image of the current block. Exemplarily, the method by which the decoding side determines the index of the reference image of the current block in this list of reference images RPL0 includes at least Method 1, where the coding end and the decoding side, by default, determine each reference image in the list of reference images RPL0, for example, the first reference image, as the reference image of the current block, thereby eliminating the need for the coding end to indicate the index of the reference image of the current block in the bitstream, and Method 2, where the coding end writes the index of the reference image of the current block in this list of reference images RPL0 to the bitstream, and the decoding side decodes the bitstream to obtain the index of the reference image of the current block in the list of reference images RPL0, and further obtains the reference image of the current block based on that index.

[0156] In some embodiments of the present invention, when the prediction of the current block is bidirectional, the current block corresponds to two reference image lists, denoted as RPL0 and RPL1, respectively. The decoding side determines the index refIdxL0 that one reference image of the current block has in reference image list RPL0, and the index refIdxL1 that the other reference image has in reference image list RPL1. Based on these two indices, the decoding side determines the two reference images of the current block from reference image list RPL0 and reference image list RPL1. Exemplary, a method by which the decoding side determines the index of the current block's reference image in the reference image list RPL0 includes at least one of the following: Method 1, the encoding end and the decoding side, by default, determine each reference image in the reference image list RPL0, for example, the first reference image, as the reference image of the current block, thereby eliminating the need for the encoding end to indicate the index of the current block's reference image in the bitstream; and Method 2, the encoding end writes the index of the current block's reference image in the reference image list RPL0 to the bitstream, thereby allowing the decoding side to decode the bitstream to obtain the index of the current block's reference image in the reference image list RPL0, and further obtain the current block's reference image based on that index.

[0157] Based on the steps described above, the decoding side determines the reference image of the current block. Since all reference images are already decoded, their motion information is known. Therefore, the decoding side can directly obtain the motion information of the reference image, and further determines the second motion information of the current block based on the motion information of the reference image and the first motion information of the current block determined in the steps described above. Here, the second motion information is understood to be more accurate motion information obtained after improving the first motion information.

[0158] In the embodiments of the present invention, there are at least two roles that the motion information of the reference image plays in improving the first motion information. One is when the motion information of the reference image directly participates in improving the first motion information, for example, when it is used to guide the search process for the second motion information. The other is when, in the process of improving the first motion information, the motion information of the reference image is used to instruct the division of blocks.

[0159] The following describes the process by which the decoding side improves the first motion information based on the motion information of the current block's reference image and obtains the second motion information for the current block.

[0160] Embodiments of the present invention do not limit the specific process by which the decoding side improves the first motion information based on the motion information of the reference image and obtains the second motion information of the current block.

[0161] In Case 1, if the motion information of the reference image directly participates in improving the first motion information, the decoding side achieves improvement of the first motion information and acquires the second motion information by at least one of the methods shown in the following embodiments.

[0162] In some embodiments, a template reference region corresponding to the current block's template is determined within the reference image. Here, the motion information of the template reference region is known, and the motion information of the current block's template is also known. Therefore, based on the motion information of the template reference region and the motion information of the current block's template, the first motion information of the current block can be improved, and the second motion information of the current block can be obtained. For example, the difference value between the motion information of the template reference region and the motion information of the current block's template can be determined, and this difference value can be added to the first motion information to obtain the second motion information of the current block.

[0163] In some embodiments, the above-described S102 includes the following steps S102-A to S102-C. S102-A, based on the first motion information, determines the reference block corresponding to the current block in the reference image. S102-B, based on the current block and according to the motion information of the reference block, the time-domain motion information of the current block is determined as the third motion information. S102-C: The first motion information is improved based on the third motion information, and the second motion information is obtained.

[0164] In this embodiment, the decoding side first determines the reference block corresponding to the current block in the reference image based on the first motion information. Since the motion information of the reference block is known, the decoding side can determine whether the first motion information of the current block is accurate based on the motion information of the reference block. For example, assume that the reference block and the current block belong to the same moving object in the image, and that the motion of the entire object is uniform. In this case, the reference block can be treated as the same block as the current block, and the time-domain motion information of the current block can be determined based on the motion information of the reference block. For the sake of simplicity, this time-domain motion information will be referred to as the third motion information. Furthermore, the first motion information is improved based on this third motion information, thereby achieving an accurate improvement of the first motion information.

[0165] As can be seen from the above, the first motion information currently includes information on the motion vectors of the block. In the embodiments of the present invention, improvements to the first motion information are understood to be improvements to the motion vectors included in the first motion information.

[0166] Specifically, the decoding side first determines the reference block corresponding to the current block in the reference image based on the first motion information. Here, the first motion information of the current block may be either unidirectional or bidirectional motion information. The following describes each of these two cases.

[0167] In one example, if the current block uses unidirectional prediction, the first motion information of the current block includes unidirectional motion information. That is, the current block corresponds to one reference image and one motion vector. Assume that the reference image included in the first motion information of the current block curr_block in the current image curr_pic is the reference image ref_pic_0 in the reference image sequence RPL0, and the motion vector is the first motion vector mv_0. Assume that the playback order of reference image ref_pic_0 is earlier than that of the current image curr_pic. Thus, as shown in Figure 15, the decoding side can position the corresponding reference block ref_block_0 within reference image ref_pic_0 based on the position of the current block curr_block and the first vector mv_0 of the current block. In Figure 15, the current block represented by the dotted frame in the reference image is understood to be the same block in the reference image.

[0168] In one example, if the current block uses bidirectional prediction, the first motion information of the current block includes bidirectional motion information. That is, the current block corresponds to two reference images (denoted as the first reference image and the second reference image) and two motion vectors (denoted as the first motion vector and the second motion vector). Assume that the first reference image included in the first motion information of the current block curr_block in the current image curr_pic is the reference image ref_pic_0 from the reference image sequence RPL0, and the first motion vector is mv_0, and the second reference image of the current block is the reference image ref_pic_1 from the reference image sequence RPL1, and the second motion vector is mv_1. Assume that the playback order of the first reference image ref_pic_0 is before the current image curr_pic, and the playback order of the second reference image ref_pic_1 is after the current image curr_pic. Thus, as shown in Figure 16, the decoding side can position the corresponding first reference block ref_block_0 in the first reference image ref_pic_0 based on the current position of the block curr_block and the first motion vector mv_0, and can position the corresponding second reference block ref_block_1 in the second reference image ref_pic_1 based on the current position of the block curr_block and the second motion vector mv_1.

[0169] Based on the steps described above, the decryption process determines the reference block that the current block corresponds to within the reference image, based on the first motion information of the current block. Since all reference images are decrypted images, their motion information is known, and therefore the motion information of the reference block can also be obtained.

[0170] In the embodiment of the present invention, the decoding side can infer, based on the motion information of the reference block, which direction the first motion vector of the current block is most likely to shift. Specifically, based on the motion information of the reference block, the time-domain motion information of the current block is determined and denoted as the third motion information. This third motion information is compared with the first motion information of the current block to determine which direction the first motion information is most likely to shift.

[0171] The implementation method for determining the third motion information in S102-B described above is not limited to the following.

[0172] Method 1: The decoding side treats the current block as an equivalent block to the reference block, determines the time-domain motion information of the reference block based on the motion information of the reference block, and further determines the reciprocal of this time-domain motion information as the third motion information.

[0173] In one example, assuming that the current block uses unidirectional prediction, we assume that the motion within each reference block is the same. Based on the motion information of the reference block, the decoding side can infer the vector information in the time-domain motion information when the reference block moves from its current position in the reference image towards the current block in the current image, and we denote this as mv_t. For example, the decoding side derives mv_t using the method for deriving time-domain motion information. Furthermore, the inverse vector of this mv_t, -mv_t, is determined as the motion vector in the third motion information.

[0174] For example, the motion vector mv_0_t of the time-domain motion information of the reference block is determined by the following equation (10).

number

[0175] Assume that both the reference block and the current block belong to the same object undergoing global motion, that the motion of this object is uniform linear motion, and that the motion information of the reference block is accurate. Compare the absolute value of the motion vector -mv_t in the third motion information determined above with the absolute value of the motion vector mv_0 in the first motion information of the current block to determine in which direction the first motion information is likely to shift. For example, if the absolute value of the motion vector -mv_t in the third motion information is smaller than the absolute value of the motion vector mv_0 in the first motion information of the current block, it indicates that the reference block can only reach the dotted line frame position shown in Figure 17A with its current motion, and it can be inferred that the motion vector mv_0 of the current block in the first motion information is too large. In another example, if the absolute value of the motion vector -mv_t in the third motion information is greater than the absolute value of the motion vector mv_0 in the first motion information of the current block, it can be shown that the reference block can reach the dotted frame position shown in Figure 17B with its current motion, and it can be inferred that the motion vector mv_0 of the current block in the first motion information is too small.

[0176] In one example, assuming that the current block uses bidirectional prediction, we can assume that the motion within each reference block is the same. Based on the motion information of the first reference block, the decoding side can infer motion vector 0 from the first reference image to the current image, denoted as mv_0_t. Similarly, based on the motion information of the second reference block, the decoding side can infer motion vector 1 from the second reference image to the current image, denoted as mv_1_t. Here, motion vectors 0 and 1 can be derived using the method for deriving time-domain motion information.

[0177] For example, the motion vector mv_0_t of the time-domain motion information of the first reference block is determined by the following equation (11).

number

[0178] For example, the motion vector mv_1_t of the time-domain motion information of the second reference block is determined by the following equation (12).

number

[0179] Next, the decoding side determines the inverse vector of mv_0_t, -mv_0_t, as the first predicted direction motion vector in the third motion information, and the inverse vector of mv_1_t, -mv_1_t, as the second predicted direction motion vector in the third motion information.

[0180] Assume that both the reference block and the current block belong to the same globally moving object, that the motion of this object is uniform linear motion, and that the motion information of the reference block is accurate. Compare the absolute value of the motion information in the third motion information determined above with the absolute value of the motion vector in the first motion information of the current block to determine in which direction the first motion information is most likely to shift. Specifically, since the current block in the embodiment of this application uses bidirectional prediction, it can be determined that both the first motion information and the third motion information include a first predicted direction motion vector and a second predicted direction motion vector. Then, compare the motion information in each direction. First, compare the absolute value of the first predicted direction motion vector -mv_0_t in the third motion information with the absolute value of the first predicted direction motion vector mv_0 in the first motion information. For example, if the absolute value of the first predicted directional motion vector -mv_0_t in the third motion information is smaller than the absolute value of the first predicted directional motion vector mv_0 in the first motion information, it indicates that when the first reference block moves from its current position in the first reference image toward the current image using its current motion vector, the current block will not be able to reach its position in the current image, and it can be inferred that the first predicted directional motion vector mv_0 in the first motion information is too large. In another example, if the absolute value of the first predicted directional motion vector -mv_0_t in the third motion information is larger than the absolute value of the first predicted directional motion vector mv_0 in the first motion information, it indicates that when the first reference block moves from its current position toward the current image using its current motion vector, its position in the current image will exceed the current block's position in the current image, and it can be inferred that the first predicted directional motion vector mv_0 in the first motion information is too small. Next, we compare the absolute value of the second predicted direction motion vector -mv_1_t in the third motion information with the absolute value of the second predicted direction motion vector mv_1 in the first motion information.For example, if the absolute value of the second predicted directional motion vector -mv_1_t in the third motion information is smaller than the absolute value of the second predicted directional motion vector mv_1 in the first motion information, it indicates that when the second reference block moves from its current position in the second reference image toward the current image using its current motion vector, the current block will not be able to reach its current position in the current image, and it can be inferred that the second predicted directional motion vector mv_0 in the first motion information is too large. In another example, if the absolute value of the second predicted directional motion vector -mv_1_t in the third motion information is larger than the absolute value of the second predicted directional motion vector mv_1 in the first motion information, it indicates that after the second reference block moves from its current position in the second reference image toward the current image using its current motion vector, its position in the current image will exceed the current block's position in the current image, and it can be inferred that the second predicted directional motion vector mv_1 in the first motion information is too small.

[0181] Method 2: The decoding side determines the time-domain motion information of the current block as the third motion vector based on the motion information of the reference block.

[0182] In one example, assuming that the current block uses unidirectional prediction, it is assumed that the motion within each reference block is the same. The decoding side determines the time-domain motion information of the current block as the third motion information based on the motion information of the reference block.

[0183] For example, the third motion information is determined by the following equation (13).

number

[0184] Assume that both the reference block and the current block belong to the same object undergoing global motion, that the motion of this object is uniform linear motion, and that the motion information of the reference block is accurate. Compare the absolute value of the motion vector mv_t' in the third motion information determined above with the absolute value of the motion vector mv_0 in the first motion information of the current block to determine in which direction the first motion information is likely to shift. For example, if the absolute value of the motion vector mv_t' in the third motion information is smaller than the absolute value of the motion vector mv_0 in the first motion information of the current block, it indicates that when the current block moves from the current position where the current image is located toward the reference image with its current motion vector, it can only reach the dotted line frame position shown in Figure 18A, and it can be inferred that the motion vector mv_0 of the current block in the first motion information is too small. In another example, if the absolute value of the motion vector mv_t' in the third motion information is greater than the motion vector mv_0 in the first motion information of the current block, we can show that when the current block moves from its current position where the current image is located toward the reference image using its current motion vector, it can reach the dotted frame position shown in Figure 18B, and we can infer that the motion vector mv_0 of the current block in the first motion information is too large.

[0185] In one example, if the current block uses bidirectional prediction, both the first and third motion information contain the first and second predicted direction motion vectors. Assume that the motion within each reference block is the same. Based on the motion information of the first reference block, the decoder can estimate the motion vector mv_0_t' when the current block moves from its current position where the current image is located towards the first reference image. The decoder can also estimate the motion vector mv_1_t' when the current block moves from its current position where the current image is located towards the second reference image, based on the motion information of the second reference block. Here, mv_0_t' is the first predicted direction motion vector in the third motion information, and mv_1_t' is the second predicted direction motion vector in the third motion information. Here, mv_0_t' and mv_1_t' can be derived using the method for deriving time-domain motion information.

[0186] For example, the first predicted direction motion vector mv_0_t' in the third motion information is determined by the following equation (14).

number

[0187] For example, the second predicted direction motion vector mv_1_t' in the third motion information is determined by the following equation (15).

number

[0188] Assume that both the reference block and the current block belong to the same object undergoing global motion, that the motion of this object is uniform linear motion, and that the motion information of the reference block is accurate. Compare the absolute value of the motion vector in the third motion information determined above with the absolute value of the motion vector in the first motion information of the current block to determine in which direction the first motion information is most likely to shift. Specifically, first compare the absolute value of the first predicted direction motion vector mv_0_t' in the third motion information with the absolute value of the first predicted direction motion vector mv_0 in the first motion information. For example, if the absolute value of the first predicted directional motion vector mv_0_t' in the third motion information is smaller than the absolute value of the first predicted directional motion vector mv_0 in the first motion information, it indicates that when the current block moves from its current position in the current image toward the first reference image according to the first predicted directional motion vector mv_0, the current first reference block will not be able to reach its position in the first reference image, and it can be inferred that the first predicted directional motion vector mv_0 in the first motion information is too large. In another example, if the absolute value of the first predicted directional motion vector mv_0_t' in the third motion information is larger than the absolute value of the first predicted directional motion vector mv_0 in the first motion information, it indicates that after the current block moves from its current position in the current image toward the first reference image according to the first predicted directional motion vector mv_0, its position in the first reference image will exceed the current position of the first reference block in the first reference image, and it can be inferred that the first predicted directional motion vector mv_0 in the first motion information is too small. Next, we compare the absolute value of the second predicted directional motion vector mv_1_t' in the third motion information with the absolute value of the second predicted directional motion vector mv_1 in the first motion information. For example, if the absolute value of the second predicted directional motion vector mv_1_t' in the third motion information is smaller than the absolute value of the second predicted directional motion vector mv_1 in the first motion information, it indicates that when the current block moves from its current position in the current image toward the second reference image according to the second predicted directional motion vector mv_0, the current second reference block cannot reach its current position in the second reference image, and we can infer that the second predicted directional motion vector mv_0 in the first motion information is too large.In another example, if the absolute value of the second predicted directional motion vector mv_1_t' in the third motion information is greater than the absolute value of the second predicted directional motion vector mv_1 in the first motion information, it indicates that after the current block moves from its current position in the current image toward the second reference image according to the second predicted directional motion vector mv_0, its position in the second reference image will exceed the current position of the second reference block in the second reference image, and it can be inferred that the second predicted directional motion vector mv_1 in the first motion information is too small.

[0189] Based on the steps described above, the decoding side determines the third motion information and then executes the steps of S102-C described above.

[0190] In some embodiments, improving the first motion information using the third motion information described above may result in a larger error, for example, when the object is not moving at a constant velocity. Therefore, in embodiments of the present application, the decoding side first determines the difference between the first motion information and the third motion information before improving the first motion information based on the third motion information and obtaining the second motion information.

[0191] The embodiments of this invention do not limit the specific method by which the decoding side determines the difference value between the first motion information and the third motion information.

[0192] In one possible implementation method, the absolute value of the difference between the motion vector in the first motion information and the motion vector in the third motion information is determined as the difference value between the first motion information and the third motion information.

[0193] For example, the difference value may be determined by the following equation (16).

number

[0194] If this difference value is greater than a predetermined threshold thr, the step of improving the first motion information based on the third motion information, i.e., obtaining the second motion information, is skipped, and instead the first motion information is improved directly using a method such as DMVR, and the second motion information is obtained. If this difference value is less than or equal to the predetermined threshold, the steps of S102-C described above are executed, the first motion information is improved based on the third motion information, and the second motion information is obtained.

[0195] The embodiments of this application do not limit the specific method by which the decoding side improves the first motion information based on the third motion information to obtain the second motion information.

[0196] In some embodiments, the decoding side compares the third motion information with the first motion information, adjusts the first motion information based on the third motion information, and obtains improved second motion information. For example, if the first motion information is determined to be smaller than the third motion information, the first motion information may be adaptively increased to obtain the second motion information. For example, when searching for the second motion information around the first motion information, there may be a stronger tendency to search for a larger motion vector. Similarly, if the first motion information is determined to be larger than the third motion information, the first motion information may be adaptively decreased to obtain the second motion information. For example, when searching for the second motion information around the first motion information, there may be a stronger tendency to search for a smaller motion vector.

[0197] In some embodiments, the above-described S102-C includes the following steps S102-C1 and S102-C2. S102-C1, the fourth motion information is determined based on the third motion information and the first motion information. S102-C2, the second motion information is determined based on the fourth motion information.

[0198] In this implementation method, the method by which the decoding side determines the fourth motion information based on the third motion information and the first motion information is not limited to the following few methods.

[0199] Method 1: The average value of the third motion information and the first motion information is determined as the fourth motion information.

[0200] For example, if the current block uses unidirectional prediction and the first, second, and fourth motion information all include unidirectional motion information, the decoding side may determine the fourth motion information using the following equation (17).

number

[0201] For example, if the current block uses bidirectional prediction and the first motion information, second motion information, and fourth motion information include the first predicted direction motion vector and the second predicted direction motion vector, the decoding side may determine the fourth motion information by the following equation (18).

number

[0202] Method 2 determines the weights corresponding to the third motion information and the first motion information, determines the weighted average of the third motion information and the first motion information based on the weights, and further determines this weighted average as the fourth motion information.

[0203] The embodiments of this application do not limit the specific method by which the decoding side determines the weights corresponding to the third motion information and the first motion information.

[0204] In one example, the weight corresponding to the third motion information is greater than the weight corresponding to the first motion information.

[0205] In one example, the weight corresponding to the third motion information is smaller than the weight corresponding to the first motion information. This is because the first motion information is a kind of motion information assumption (or prediction) determined based on relevant prediction information provided by the encoder, whereas the relevant prediction information provided by the encoder includes motion information that the encoder has selected from a list of candidates and deemed appropriate, or a prediction mode that the encoder has selected and deemed appropriate. The third motion information is a kind of motion information assumption (or prediction) inferred by the decoder. For example, it is the motion information of the current block derived by the decoder based on the motion information on the reference image. In this example, the first motion information is obtained through selection by the encoder, and the third motion information is derived based on the motion information of the reference image, but since the reference image is not the current image, a relatively high weight can be set for the first motion information and a relatively low weight for the third motion information.

[0206] The decoding side determines the weights corresponding to the third motion information and the first motion information, then performs a weighting process on the third motion information and the first motion information to obtain the fourth motion information.

[0207] For example, if the current block uses unidirectional prediction and the first, second, and fourth motion information all include unidirectional motion information, the decoding side may determine the fourth motion information using the following equation (19).

number

[0208] In one example, b0 = 1 - a0.

[0209] For example, a0 could be 3 / 4, 5 / 8, etc., and b0 could be 1 / 4, 2 / 8, etc.

[0210] For illustrative purposes, to avoid the appearance of decimals or fractions, if a is 3 / 4, the above equation (19) can also be written in the form of equation (20) below.

number

[0211] In some embodiments, the division in the above formula may be replaced with a right shift >>.

[0212] For example, if the current block uses bidirectional prediction and the first motion information, second motion information, and fourth motion information include the first predicted direction motion vector and the second predicted direction motion vector, the decoding side may determine the fourth motion information by the following equation (21).

number

[0213] In some embodiments, different weights may be set depending on the specific situation. For example, different weights may be set depending on the difference in prediction modes.

[0214] Based on the steps described above, the decoding side determines the fourth motion information, then executes S102-C2 described above, and determines the second motion information based on this fourth motion information.

[0215] In the embodiments of the present application, the specific implementation method by which the decoding side determines the second motion information based on the fourth motion information in S102-C2 described above is not limited to the following.

[0216] Method 1: The search center for the second motion information is positioned at the location corresponding to the fourth motion information. At this time, the above-mentioned S102-C2 includes the following step S102-C2-a.

[0217] S102-C2-a, the corresponding position in the reference image of the fourth motion information is used as the search center point for the second motion information, and the second motion information is obtained by searching within the reference image.

[0218] In the embodiments of this invention, the second motion information is obtained by searching around the first motion information. For example, as shown in Figure 19A, the decoding side determines the positioning point of the current block's reference block in the reference image based on the first motion information. This positioning point may be the upper left corner of the reference block or the center of the reference block. Next, the vicinity of the positioning point specified by this first motion information is searched. For example, the search is centered on that positioning point, or the same range is searched around that positioning point in all directions (up, down, left, and right). Exemplarily, in Figure 19A, each square represents a single pixel position, the black squares are the positioning points specified by the first motion information in the reference image, and the white squares are the search positions for the second motion information. Of these search points in the squares, the motion information corresponding to the position with the minimum cost is determined as the second motion information.

[0219] As can be seen from the above, there is a possibility that the first motion information in the embodiment of the present invention may have an inaccurate problem. For example, if the first motion information is determined to be too large or too small based on the motion information of the reference image, searching for the second motion information using the inaccurate first motion information as the search center will result in an inaccurate search for the second motion information. To solve this technical problem, the embodiment of the present invention determines the third motion information based on the motion information of the reference image, and further modifies the search center for the second motion information based on the third motion information and the first motion information of the current block. Specifically, the fourth motion information is determined based on the third motion information and the first motion information, and the second motion information is obtained by searching the reference image using the corresponding position in the reference image of the fourth motion information as the search center point for the second motion information. Since this fourth motion information takes into account the motion information of the reference image and the first motion information of the current block, by setting the position specified by the fourth motion information as the search center for the second motion information, the accuracy of the search center can be improved, further improving the accuracy of the search for the second motion information and improving the decoding prediction effect.

[0220] Let's illustrate with an example. The analysis above indicates that if the first motion information is larger than the third motion information, the first motion information is too large. In this case, the search for the second motion information tends to focus on smaller motion information. As shown in Figure 19B, the black squares represent the positions specified by the first motion information within the reference image, and the gray squares represent the positions specified by the third motion information within the reference image. If we set the search center for the second motion information to the corresponding positions in the reference image of the average of the third and first motion information (i.e., the fourth motion information), the resulting search range is as shown in Figure 19B. Compared to Figure 19A, the search range shifts to the right, biasing towards searching for smaller motion information and achieving accurate searching of the second motion information. In another example, if the first motion information is smaller than the third motion information, the first motion information is too small. In this case, the search for the second motion information tends to focus on larger motion information. For example, as shown in Figure 19C, the search range for the second motion information shifts to the left compared to Figure 19A, biasing towards searching for larger motion information and enabling accurate searching of the second motion information.

[0221] In the embodiments of the present invention, the specific process for determining the search range of the second motion information is essentially the same whether the current block is using unidirectional or bidirectional prediction.

[0222] In some embodiments, when the current block uses unidirectional prediction, the process of obtaining the second motion information by searching within the reference image, with the corresponding position in the reference image of the fourth motion information as the search center point for the second motion information, may refer to the methods shown in Figures 19B and 19C. For example, the corresponding position in the reference image of the fourth motion information is used as the search center point for the second motion information, motion information is searched within a predetermined search range near that search center point in the reference image, the cost of the motion information corresponding to each searched position is determined, and the motion information corresponding to the position with the minimum cost is determined as the second motion information of the current block. In unidirectional prediction, the cost of the motion information corresponding to each position can be expressed as the matching cost between the motion information of the template for that position and the motion information of the template for the current block.

[0223] In some embodiments, when the current block uses bidirectional prediction, the current block includes a first reference image and a second reference image, and both the second motion information and the fourth motion information include first predicted direction motion information and second predicted direction motion information. Thus, the decoding side uses the corresponding position in the first reference image of the first predicted direction motion information in the fourth motion information as the search center point for the first predicted direction motion information in the second motion information, and the corresponding position in the second reference image of the second predicted direction motion information in the fourth motion information as the search center point for the second predicted direction motion information in the second motion information, performs motion information retrieval within a predetermined search range of the first and second reference images, determines the bidirectional matching cost for each pair of bidirectional motion information retrieved (each pair of motion information includes one first predicted direction motion information and one second predicted direction motion information), and further determines the second motion information from the multiple pairs of bidirectional motion information retrieved based on the bidirectional matching cost.

[0224] In this bidirectional prediction, it is assumed that the predetermined search ranges on both sides are the same and both contain n possible search locations. In other words, n possible MVs can be searched on each side.

[0225] In one possible implementation, the decoding side, during the search, combines two pairs of n possible MVs corresponding to the first reference image and two pairs of n possible MVs corresponding to the second reference image, and n 2 It is possible to obtain paired bidirectional motion information.

[0226] In one possible implementation, when searching for bidirectional motion information between the first and second reference images, the MVs of the two reference images move mirror-image-like during movement. That is, based on the MV corresponding to each search center point, one moves by MVdiff and the other moves by -MVdiff. In this case, n pairs of bidirectional motion information can be obtained.

[0227] When the bidirectional motion information for each pair of the aforementioned pairs of bidirectional motion information is searched and obtained, the bidirectional matching cost between that bidirectional motion information is determined.

[0228] The embodiments of this invention do not limit the specific method for determining the bidirectional matching cost between bidirectional motion information.

[0229] Taking the i-th pair of bidirectional motion information in a set of multiple pairs as an example, in one possible implementation, this i-th pair of bidirectional motion information includes a first predicted direction motion information (e.g., a first predicted direction motion vector MV0) and a second predicted direction motion information (e.g., a second predicted direction motion vector MV1). Since both MV0 and MV1 are vectors, the distance between MV0 and MV1 can be determined based on a vector distance formula, and this distance can then be determined as the bidirectional matching cost corresponding to the i-th pair of bidirectional motion information.

[0230] In one possible implementation, the decoding side can determine a first predicted block in the first reference image based on the first predicted direction motion information in the i-th pair of bidirectional motion information, and determine a second predicted block in the second reference image based on the second predicted direction motion information in the i-th pair of bidirectional motion information. The matching cost of the first and second predicted blocks is determined, and the bidirectional matching cost of the i-th pair of bidirectional motion information is determined based on the matching cost of the first and second predicted blocks. For example, the SAD cost of the first and second predicted blocks is determined as the matching cost of the first and second predicted blocks, and this matching cost is further determined as the bidirectional matching cost of the i-th pair of bidirectional motion information, or the bidirectional matching cost of the i-th pair of bidirectional motion information is obtained by multiplying or dividing this matching cost by a predetermined coefficient.

[0231] Based on the steps described above, the decoding side can determine the bidirectional matching cost for each pair of the searched pairs of bidirectional motion information, and further, it determines the pair of bidirectional motion information with the minimum bidirectional matching cost among the searched pairs of bidirectional motion information as the second motion information. The obtained second motion information is bidirectional motion information.

[0232] In Method 1 described above, the corresponding position in the reference image of the fourth motion information was used as the search center point for the second motion information, and the specific process of obtaining the second motion information by searching within the reference image was explained. Method 2 will be explained below.

[0233] Method two, the cost in the second motion information retrieval process is corrected through the fourth motion information. At this time, the above S102-C2 includes the following steps S102-C2-b1 to S102-C2-b4. In S102-C2-b1, the corresponding position in the reference image of the first motion information is used as the search center point for the second motion information, and motion information is searched within the reference image to determine the first cost of each candidate motion information found. S102-C2-b2, based on candidate movement information and fourth movement information, the cost coefficient corresponding to the candidate movement information is determined. S102-C2-b3. Modify the first cost based on the cost coefficient corresponding to the candidate motion information to obtain the second cost of the candidate motion information. S102-C2-b4. Determine the second motion information based on the second costs of the multiple candidate motion information searched.

[0234] In this second method, based on the current first motion information, the first cost of each candidate motion information is searched and determined. Next, the cost coefficient determined by the fourth motion information is used to modify the first cost of the candidate motion information to obtain the second cost. Furthermore, based on the second costs of each candidate motion information, the second motion information is selected from each candidate motion information.

[0235] Specifically, the decoding side first uses the corresponding position in the reference image of the first motion information as the search center point for the second motion information, performs motion information search within the reference image, and determines the first cost of each candidate motion information searched. For example, referring to FIG. 19A described above, the black square is the positioning point specified by the first motion information within the reference image. Using this positioning point as the search center for the second motion information, the motion information at the position corresponding to the white square is recorded as the candidate motion information for the second motion information, and the first cost of each of these candidate motion information is determined.

[0236] Next, for each candidate motion information searched above, based on the candidate motion information and the fourth motion information, determine the cost coefficient corresponding to the candidate motion information. For example, determine the absolute value of the difference between the candidate motion information and the fourth motion information as the cost coefficient corresponding to the candidate motion information. Alternatively, determine the sum of the absolute value of the difference between the candidate motion information and the fourth motion information and a predetermined value as the cost coefficient corresponding to the candidate motion information.

[0237] In this way, the first cost of the candidate motion information can be modified by the cost coefficient determined above to obtain the second cost. For example, determine the product of the cost coefficient and the first cost of the candidate motion information as the second cost of the candidate motion information.

[0238] In the embodiments of the present application, when the current block uses uni - directional prediction or bi - directional prediction, the specific process of determining the search range of the second motion information is basically the same.

[0239] In some embodiments, when the current block uses uni - directional prediction, both the first motion information and the candidate motion information are uni - directional motion information. As shown in FIG. 19A, the decoding side first uses the corresponding position in the reference image of the first motion information as the search center point of the second motion information, performs motion information search within a predetermined search range of the reference image, and determines the first cost of each searched candidate motion information. In uni - directional prediction, the first cost of each candidate motion information can be represented by the matching cost between the motion information of the template at the position where the candidate motion information is located and the motion information of the template of the current block. Next, based on each candidate motion information and the fourth motion information, a cost coefficient corresponding to each candidate motion information is determined. For example, the absolute value of the difference between the candidate motion information and the fourth motion information is determined as the cost coefficient corresponding to the candidate motion information. Alternatively, the sum of the absolute value of the difference between the candidate motion information and the fourth motion information and a predetermined value is determined as the cost coefficient corresponding to the candidate motion information. Then, the first cost is corrected based on the cost coefficient corresponding to the candidate motion information to obtain the second cost of the candidate motion information. For example, the product of the cost coefficient and the first cost of the candidate motion information is determined as the second cost of the candidate motion information. In this way, the second costs of a plurality of searched candidate motion information can be determined, and further, the candidate motion information with the minimum second cost among the plurality of candidate motion information is determined as the second motion information. This second motion information is uni - directional motion information.

[0240] In some embodiments, when the current block uses bi - directional prediction, the above - mentioned current block includes a first reference image and a second reference image, and the first motion information, the second motion information, the fourth motion information, and the candidate motion information all include first - prediction - direction motion information and second - prediction - direction motion information. At this time, S102 - C2 - b1 described above includes the following steps.

[0241] In S102-C2-b11, the corresponding position in the first reference image of the first predicted direction motion information in the first motion information is set as the search center point for the first predicted direction motion information in the second motion information, and the corresponding position in the second reference image of the second predicted direction motion information in the first motion information is set as the search center point for the second predicted direction motion information in the second motion information. Motion information is searched within a predetermined search range of the first and second reference images, and the first cost of each candidate motion information found is determined.

[0242] In this bidirectional prediction, it is assumed that the predetermined search ranges on both sides are the same and both contain n possible search locations. In other words, n possible MVs can be searched on each side.

[0243] In one possible implementation, the decoding side, during the search, uses the corresponding position in the first reference image of the first predicted direction motion information in the first motion information as the search center point for the first predicted direction motion information in the second motion information, and can search for n possible MVs on the first reference image side. The corresponding position in the second reference image of the second predicted direction motion information in the first motion information is used as the search center point for the second predicted direction motion information in the second motion information, and can search for n possible MVs on the second reference image side. Combining the n possible MVs on both sides in pairs, n 2 By obtaining paired bidirectional motion information, further n 2 Obtain individual candidate movement information. Each candidate movement information is bidirectional movement information.

[0244] In one possible implementation, the corresponding position in the first reference image of the first predicted direction motion information in the first motion information is set as the search center point for the first predicted direction motion information in the second motion information, and the corresponding position in the second reference image of the second predicted direction motion information in the first motion information is set as the search center point for the second predicted direction motion information in the second motion information. When searching for candidate motion information within a predetermined search range of the first and second reference images, the MVs of the two reference images move mirror-image. That is, based on the MV corresponding to each search center point, one moves by MVdiff and the other moves by -MVdiff. At this time, n pairs of bidirectional motion information can be obtained, and further n candidate motion information can be obtained. Each candidate motion information is bidirectional motion information.

[0245] When a candidate motion information is found among the multiple candidate motion information described above, the first cost of that candidate motion information is determined. At this time, the first cost may be a bidirectional matching cost. In the embodiments of this application, the process of determining the first cost of each candidate motion information is the same. One candidate motion information will be explained as an example.

[0246] In one possible implementation, this candidate motion information includes first predicted direction motion information (e.g., first predicted direction motion vector MV0) and second predicted direction motion information (e.g., second predicted direction motion vector MV1). Since both MV0 and MV1 are vectors, the distance between MV0 and MV1 may be determined based on a vector distance formula, and this distance may be determined as the first cost of the candidate motion information.

[0247] In one possible implementation, the decoding side determines a first predicted block in the first reference image based on the first predicted direction motion information in the candidate motion information, determines a second predicted block in the second reference image based on the second predicted direction motion information in the candidate motion information, determines the matching cost between the first and second predicted blocks, and determines the first cost of the candidate motion information based on the matching cost between the first and second predicted blocks. For example, the SAD cost of the first and second predicted blocks is determined as the matching cost between the first and second predicted blocks, and this matching cost is further determined as the first cost of the candidate motion information. Alternatively, the first cost of the candidate motion information is obtained by multiplying or dividing this matching cost by a predetermined coefficient.

[0248] Next, based on the candidate movement information and the fourth movement information, the cost coefficient corresponding to the candidate movement information is determined.

[0249] This method two does not change the search range for the second movement information, but strengthens the tendency to select candidate movement information that is close to the fourth movement information. Therefore, a coefficient may be multiplied by the first cost of each candidate movement information. For example, a relatively small cost coefficient can be set for candidate movement information that is close to the fourth movement information, and a relatively large cost coefficient can be set for candidate movement information that is far from the fourth movement information.

[0250] In the embodiments of the present invention, if the candidate motion information is bidirectional motion information, there may be one or two corresponding cost coefficients.

[0251] In some embodiments, if the candidate motion information corresponds to a single cost coefficient, the absolute value of the difference between the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information is determined and denoted as difference value 1. The absolute value of the difference between the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information is determined and denoted as difference value 2. Difference value 3 is determined based on difference value 1 and difference value 2. For example, the sum or average of difference value 1 and difference value 2 is determined as difference value 3. Furthermore, a single cost coefficient corresponding to the candidate motion information is determined based on this difference value 3. For example, this difference value 3 is determined as the cost coefficient corresponding to the candidate motion information. Alternatively, the sum of this difference value 3 and a predetermined value is determined as the cost coefficient corresponding to the candidate motion information. The second cost of the candidate motion information can be determined using the single cost coefficient determined in this way and the first cost described above. For example, the product of the first cost of the candidate motion information and the single cost coefficient corresponding to that candidate motion information is determined as the second cost of the candidate motion information.

[0252] In some embodiments, when the candidate motion information corresponds to two cost coefficients, namely a first cost coefficient and a second cost coefficient, the above-described S102-C2-b2 includes the following steps. S102-C2-b21, based on the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information, the first cost coefficient corresponding to the first predicted direction motion information in the candidate motion information is determined. S102-C2-b21, based on the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information, the second cost coefficient corresponding to the second predicted direction motion information in the candidate motion information is determined.

[0253] In this embodiment, if the cost coefficient corresponding to the candidate motion information is two cost coefficients, the decoding side determines the first cost coefficient corresponding to the first predicted direction motion information in the candidate motion information based on the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information, and determines the second cost coefficient corresponding to the second predicted direction motion information in the candidate motion information based on the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information.

[0254] In the embodiments of this application, the process by which the decoding side determines the first cost coefficient and the process by which it determines the second cost coefficient are essentially the same.

[0255] In one possible implementation method, the absolute value of the difference between the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information is determined as the first cost coefficient corresponding to the first predicted direction motion information in the candidate motion information. The absolute value of the difference between the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information is determined as the second cost coefficient corresponding to the second predicted direction motion information in the candidate motion information.

[0256] In one possible implementation, the absolute value of the difference between the i-th predicted direction motion information in the candidate motion information and the i-th predicted direction motion information in the fourth motion information is determined. i is either one or two. Based on the absolute value of the difference, the i-th cost coefficient is determined. Here, the i-th cost coefficient has a negative correlation with the absolute value of the difference. That is, the absolute value of the difference of 1 between the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information is determined, and the first cost coefficient is determined based on this absolute value of 1. This first cost coefficient has a negative correlation with the absolute value of 1, and the larger the distance of 1, the smaller the first cost coefficient. The absolute value of the difference of 2 between the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information is determined, and the second cost coefficient is determined based on this absolute value of 2. This second cost coefficient has a negative correlation with the absolute value of 2, and the larger the absolute value of 2, the smaller the second cost coefficient.

[0257] Embodiments of the present application do not limit the specific method for determining the i-th cost coefficient based on the absolute value of the above difference.

[0258] In one example, the absolute value of this difference is determined as the i-th cost coefficient. That is, the absolute value 1 of the difference between the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information is determined as the first cost coefficient. The absolute value 2 of the difference between the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information is determined as the second cost coefficient.

[0259] In another example, the minimum value between the absolute value of the difference and the first predetermined value is determined. Based on the minimum value, the i-th cost coefficient is determined. That is, the absolute value 1 of the difference between the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information is compared with the first predetermined value, and the minimum value 1 between the absolute value 1 of the difference and the first predetermined value is determined. Further, based on this minimum value 2, the first cost coefficient is determined. And, the absolute value 2 of the difference between the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information is compared with the first predetermined value, and the minimum value 2 between the absolute value 2 of the difference and the first predetermined value is determined. Further, based on this minimum value 2, the second cost coefficient is determined.

[0260] Embodiments of the present application do not limit the specific value of the first predetermined value described above.

[0261] Exemplarily, the first predetermined value is a numerical value greater than 0.

[0262] Optionally, the first predetermined value is 4.

[0263] Embodiments of the present application do not limit the specific method for determining the i-th cost coefficient based on the minimum value.

[0264] For example, this minimum value is determined as the i-th cost coefficient. For example, the above minimum value 1 is determined as the first cost coefficient, and the minimum value 2 is determined as the second cost coefficient.

[0265] In another example, the sum of the minimum value and the second predetermined value is determined as the i-th cost coefficient. For example, the sum of the minimum value 1 and the second predetermined value is determined as the first cost coefficient, and the sum of the minimum value 2 and the second predetermined value is determined as the second cost coefficient.

[0266] For example, the decryption side determines the distinct first and second cost coefficients using the following equation (21).

number

[0267] The embodiments of this application do not limit the specific values ​​of the first predetermined value a and the second predetermined value b described above.

[0268] Selectively, the first predetermined value a is 4.

[0269] Selectively, the second predetermined value b is 32.

[0270] Based on the steps described above, the decoding side determines the first and second cost coefficients of the candidate movement information, and then modifies the first cost of the candidate movement information based on these first and second cost coefficients to obtain the second cost of the candidate movement information.

[0271] The embodiments of the present invention do not limit the specific method by which the decoding side modifies the first cost of the candidate motion information and obtains the second cost of the candidate motion information based on the first and second cost coefficients.

[0272] In one possible implementation method, the first cost coefficient and the second cost coefficient are added together, and then multiplied by the first cost of the candidate movement information to obtain the second cost of the candidate movement information.

[0273] In one possible implementation, the first cost of the candidate motion information is multiplied by the first cost coefficient and the second cost coefficient to obtain the second cost of the candidate motion information. For example, the decoding side obtains the second cost of the candidate motion information through the following equation (23).

number

[0274] Based on the steps described above, the decoding side can obtain the second cost for each of the multiple candidate motion pieces that have been searched. This second cost is a cost that has been corrected based on the motion information of the reference image, and its accuracy is higher than that of the first cost. Furthermore, based on the second cost of each of the multiple candidate motion pieces, the second motion piece for the current block is determined from these multiple candidate motion pieces. For example, the candidate motion piece with the smallest second cost among the multiple candidate motion pieces that have been searched is determined as the second motion piece. This enables accurate determination of the second motion piece, further improves the prediction accuracy of the current block, and enhances the decoding effect of the decoding side.

[0275] All of the embodiments described above describe a process of improving the first motion information of the entire current block, treating the current block as a single block. The methods of the embodiments described above can also be used for processing based on subblocks. For example, the current block can be divided into multiple subblocks, and the first motion information of each subblock can be improved using the same method as described above for the first motion information of the current block, to obtain the second motion information of each subblock. Furthermore, based on the second motion information of each subblock, a predicted value for each subblock can be obtained. The predicted value of each subblock constitutes the predicted value of the current block.

[0276] In the above-described embodiment, in Case 1, when the motion information of the reference image directly participates in improving the first motion information, the specific process by which the decoding side improves the first motion information and obtains the second motion information was explained.

[0277] In case 2, motion information of the reference image of the embodiment of the present invention can be used to guide the division method of the relevant blocks in the first motion information improvement process.

[0278] In some embodiments, the above-described S102 includes the following steps S102-D to S102-F. S102-D, based on the motion information of the reference image, divide the current block into at least one subblock. S102-E: For the i-th subblock of at least one subblock, the first movement information of the i-th subblock is improved, and the second movement information of the i-th subblock is obtained. i is a positive integer. S102-F, based on the second movement information of N subblocks, the second movement information of the current block is obtained.

[0279] Exemplary, as shown in Figure 20, the reference block of the current block is determined in the reference image based on the first motion information of the current block, and the motion information of the reference block is obtained. The motion vector of the gray area in the upper right corner of the reference block is clearly different from other areas. A threshold can be set, and if the difference between the two motion vectors exceeds the threshold, the difference is considered clear. Because the correlation between the current block and the reference block is strong, the motion vector of the upper left corner area of ​​the current block is also clearly different compared to other areas at this time. If the current block uses improvement for the entire block, the improvement effect is not clear. Therefore, in the embodiment of the present application, when improving the first motion information of the current block, the current block is divided into at least one subblock based on the motion information of the reference image block, and the motion information is improved for each subblock independently. This not only reduces the hardware implementation cost but also provides better flexibility when dividing into subblocks for improvement, and since each subblock can independently improve its MV, it is possible to improve the accuracy of motion information improvement to some extent.

[0280] For example, if some motion information in the current block differs from other parts, the effect of improving the motion information can be enhanced by dividing the current block into multiple sub-blocks. Since the distribution of objects in the current image and the reference image differs relatively little, in the embodiment of this application, the decoding side instructs the division of the current block based on the motion information of the reference image.

[0281] Embodiments of the present invention do not limit the specific method by which the decoding side divides the current block into at least one subblock based on motion information of the reference image.

[0282] In some embodiments, the decoding side first determines the reference block corresponding to the current block in the reference image, then randomly samples motion information of several points within the reference block, then compares the motion information of these points, divides the points with motion information into a single subblock, and further divides the reference block into at least one subblock. Next, the decoding side determines the subblock in the current block corresponding to each subblock in the reference block, and further divides the current block into at least one subblock.

[0283] In some embodiments, the above-described S102-D includes the following steps S102-D1 to S102-D4. S102-D1 determines the current block's corresponding reference block within the reference image. S102-D2: The M subblocks of the current block obtain the movement information of the corresponding M subblocks within the referenced block. M is a positive integer greater than 1. S102-D3: The movement information of the M acquired subblocks is classified, and P types of classification results are obtained. P is a positive integer less than or equal to M. Based on the classification result of S102-D4, type P, divide the current block into at least one subblock.

[0284] In this implementation, the decoding side first determines the reference block of the current block in the reference image based on the first motion information of the current block. Next, the current block is divided into M subblocks in advance. The sizes of these M subblocks may be the same or different. For example, the M subblocks may all be 16x16, 8x8, or 4x4. Alternatively, if the width and height of the current block are both 2N, the current block may be divided into 4 NxN subblocks. Alternatively, if the size of the current block is Nx2N or 2NxN, the current block may be divided into 2 NxN subblocks. Next, these M subblocks in the current block determine the corresponding M subblocks in the reference block. Since the motion information of the reference block is known, the decoding side also knows that the motion information of the M subblocks in the reference block corresponds to the M subblocks in the reference block. Because the current block and the reference block are highly correlated, the decryption unit clusters the movement information of M subblocks within the acquired reference block, obtains P classification results, and then divides the current block into at least one subblock based on these P classification results.

[0285] For example, in the case of P=1 as described above, it indicates that the difference in motion information between each region of the current block is not large, and the improvement of the first motion information using the entire block can be used. Therefore, the current block is not divided, or the current block is divided into a single subblock, i.e., the current block itself.

[0286] In another example, if P is greater than 1, the current block is divided into P subblocks based on the P classification results, each corresponding to a different subblock. Here, each subblock corresponds to one type of classification result.

[0287] Based on the steps described above, the decryption process divides the current block into at least one subblock, and then performs improvements on each of the at least one subblock individually. The improvement process for each subblock is essentially the same. For the sake of simplicity, we will use the i-th subblock as an example.

[0288] The embodiments of this invention do not limit the specific method for improving the first motion information of the i-th subblock and obtaining the second motion information of the i-th subblock.

[0289] In one possible implementation, the decoding side employs the aforementioned DMVR and / or BDOF method to improve the first motion information of the i-th subblock and obtain the second motion information of the i-th subblock.

[0290] In one possible implementation, the decoding side improves the first motion information of the i-th subblock through the motion information of the reference image. Specifically, the decoding side determines the reference block corresponding to the i-th subblock in the reference image based on the first motion information of the i-th subblock. Based on the motion information of the reference block of the i-th subblock, the time-domain motion information of the i-th subblock is determined as the third motion information corresponding to this i-th subblock. Based on the third motion information, the first motion information of the i-th subblock is improved to obtain the second motion information of the i-th subblock. The specific process of this implementation can be found by referring to the specific explanation of improving the first motion information of the current block in Case 1 above. Only the current block described above needs to be replaced with the i-th subblock. This will not be repeated here.

[0291] Based on the steps described above, the decryption side can determine the second movement information for each subblock in the current block, and further determine the second movement information for the current block based on the second movement information for at least one of these subblocks.

[0292] For example, the second movement information of at least one of these subblocks is determined as the second movement information of the current block. At this time, the second movement information of the current block includes the second movement information of each subblock within the at least one subblock. In this way, when subsequently determining the predicted value of the current block based on the second movement information of the current block, the predicted value of each subblock within the at least one subblock can be determined based on the second movement information of these at least one subblocks, and furthermore, the predicted values ​​of these at least one subblock constitute the predicted value of the current block.

[0293] In another example, the average of the second motion information of at least one of these subblocks is determined as the second motion information of the current block. In this case, the second motion information of the current block includes the motion information of the entire block.

[0294] In the above-described embodiment, the process of improving the movement information for each subblock in the current block independently was explained.

[0295] In some embodiments, the decryption side may perform multiple improvements when improving the first movement information of the current block. In this case, the above-described S102 includes the following steps S102-G. S102-G: Based on the motion information of the current block's reference image, the first motion information is improved N times to obtain the second motion information. N is a positive integer greater than 1.

[0296] The embodiments of this invention do not limit the specific improvement methods used in these N improvements.

[0297] In one possible implementation, the specific improvement method used in these N improvements is the same.

[0298] In one possible implementation, the specific improvement methods used in these N improvements are all different.

[0299] In one possible implementation, the specific improvement methods used in these N improvements are partially the same and partially different.

[0300] In some embodiments, when the decoding side improves the first motion information of the current block N times, the next improvement involves dividing the block from the previous improvement. For example, in the first improvement, the decoding side improves the first motion information of the current block based on a bidirectional matching motion vector improvement method based on the entire block. In the second improvement, the current block is divided into at least one subblock, and the motion information of each subblock of the current block after the first improvement is improved based on bidirectional matching motion vector improvement based on the subblock. Selectively, the subblock size in the second improvement may be 16x16. In the third improvement, the subblock from the second improvement is divided into at least one subblock, and the motion information after the second improvement is improved based on bidirectional optical flow motion vector improvement based on the subblock. Selectively, the subblock size in this improvement may be 8x8. Of course, further steps can be added on top of this. For example, a fourth improvement can be added. For example, the motion information after the third improvement is improved based on bidirectional optical flow motion vector improvement based on a 4x4 subblock. Selectively, further improvements may be made by employing improvement methods such as point-based bidirectional optical flow motion vector improvement. Multiple improvements optimize the motion vectors by dividing the image into multiple layers from top to bottom. In the above-mentioned multiple improvements, at least one subblock division is performed based on the motion information of the reference image, and the specific division process can be found in the related explanation in S102-D above. This enhances the rationality and accuracy of the subblock division.

[0301] In some embodiments, the above-described S102-G includes the following steps S102-G1 to S102-G4. S102-G2 improves the motion information of each subblock among at least one subblock corresponding to the j-th time, and obtains the improved motion information of at least one subblock corresponding to the j-th time. If j is equal to 1, the subblock corresponding to the j-th time is currently a block. S102-G3, based on the motion information of the reference image, each subblock of at least one subblock corresponding to the j-th time is divided into blocks to obtain at least one subblock corresponding to the (j+1)-th time. S102-G4, improve the motion information of each subblock among at least one subblock corresponding to the j+1th time, repeat the process N times, and obtain second motion information.

[0302] In this implementation method, the decoding side first improves the first motion information of the current block for the first time, obtaining motion information 1 after the first improvement corresponding to the current block. Next, based on the motion information of the reference image, the current block is divided into blocks to obtain at least one subblock 2 corresponding to the second time. The motion information of each subblock 2 among the at least one subblock 2 corresponding to the second time is improved. At this time, the motion information of subblock 2 is the motion information after the first improvement. Furthermore, the improved motion information of at least one subblock 2 corresponding to the second time is obtained. Next, based on the motion information of the reference image, subblock 2 is divided into blocks to obtain at least one subblock 3 corresponding to the third time. The motion information of each subblock 3 among the at least one subblock 3 corresponding to the third time is improved. At this time, the motion information of subblock 3 is the motion information after the second improvement. Furthermore, the improved motion information of at least one subblock 3 corresponding to the third time is obtained. Next, based on the motion information of the reference image, subblock 3 is divided into blocks to obtain at least one subblock 4 corresponding to the fourth time. The motion information of each subblock 4 among the at least one subblock 4 corresponding to the fourth time is improved. At this point, the movement information for subblock 4 is the movement information after the third improvement. Furthermore, the movement information for at least one subblock 4 corresponding to the fourth improvement is obtained. The above steps are repeated, and after performing N improvements, the second movement information for the current block is obtained.

[0303] In the embodiments of this invention, the method for dividing the current block or subblock based on motion information of the reference image is basically the same as in S102-D described above. For example, the decoding side first determines the reference block that corresponds to the second block in the reference image. This second block is the current block or the subblock corresponding to the jth time. Next, the motion information of the M subblocks of this second block is obtained for the M corresponding subblocks in the reference block. M is a positive integer greater than 1. Then, the motion information of the M subblocks obtained is classified to obtain P types of classification results. P is a positive integer less than or equal to M. Finally, the second block is divided into at least one subblock based on the P types of classification results. For example, the second block is divided into P subblocks based on the subblocks that each of the P types of classification results corresponds to. For details, please refer to the related explanation in S102-D described above, and will not be repeated here.

[0304] In the embodiments of this application, the decoding side improves the motion information of the subblock and does not limit the specific method for obtaining the motion information after the subblock improvement.

[0305] In one possible implementation, the decoding side employs the aforementioned DMVR and / or BDOF methods to improve the motion information of the subblocks.

[0306] In one possible implementation, the decoding side improves the motion information of the subblock through the motion information of the reference image. Specifically, the decoding side determines the reference block corresponding to the subblock in the reference image based on the motion information of the subblock. Based on the motion information of the reference block of that subblock, the time-domain motion information of that subblock is determined as the third motion information corresponding to that subblock. Based on the third motion information, the motion information of the subblock is improved, and the improved motion information of the subblock is obtained. For the specific process of this implementation, refer to the specific explanation of improving the first motion information of the current block in Case 1 above. All that is required is to replace the current block with its subblock. This will not be repeated here.

[0307] In the embodiments described above, the process of improving the first motion information of the current block multiple times was explained.

[0308] In some embodiments, the decoding side first determines whether the current block satisfies a motion vector improvement condition based on a predetermined block as a whole, before improving the first motion information based on the motion information of the reference image of the current block to obtain the second motion information of the current block. If it is determined that the current block satisfies this motion vector improvement condition based on the block as a whole, the first motion information is improved based on the motion information of the reference image of the current block to obtain the second motion information of the current block.

[0309] In some embodiments, if the current block does not satisfy the motion vector improvement conditions based on the entire block, the method of the embodiment of the present invention includes the following steps. Step 1: Based on the motion information of the reference image, the current block is divided into blocks to obtain multiple first subblocks. Step 2: For any of the multiple first subblocks, determine whether the first subblock satisfies the motion vector improvement condition based on the entire block. Step 3: If the first subblock does not satisfy the motion vector improvement conditions based on the entire block, the first subblock is divided into blocks based on the motion information of the reference image to obtain multiple second subblocks. Step 4: For any of the multiple second subblocks, determine whether the second subblock satisfies the motion vector improvement condition based on the entire block, and repeat this process until the divided subblock satisfies the motion vector improvement condition based on the entire block, or the size of the divided subblock satisfies the predetermined size.

[0310] In this implementation method, if the decoding side determines that the current block does not satisfy the motion vector improvement condition based on the entire block, it divides the current block into layers, and stops dividing the subblocks until the resulting subblocks satisfy the motion vector improvement condition based on the entire block, or until the resulting subblocks do not satisfy the motion vector improvement condition based on the entire block, but the size of the resulting subblocks meets a predetermined size.

[0311] The embodiments of this application do not limit the specific content of the motion vector improvement conditions based on the entire block described above.

[0312] In one possible implementation, the condition for improving the motion vector based on the entire block is that the size of the block satisfies a predetermined threshold size. This predetermined threshold could be 8x8 or 4x4, for example.

[0313] To illustrate with an example, let's assume the predetermined threshold is 4x4. If the current block size is 32x32, the current block does not satisfy the motion vector improvement condition based on the entire block. Based on the motion information of the reference image, the current block is divided into multiple first subblocks. Assuming the size of the first subblock is 8x8, the size of the first subblock does not satisfy the motion vector improvement condition based on the entire block. Next, based on the motion information of the reference image, the first subblock is divided into multiple second subblocks. Assuming the size of the second subblock is 4x4, the second subblock satisfies the motion vector improvement condition based on the entire block. Furthermore, based on the motion information of the reference image, the first motion information of each second subblock is improved to obtain the second motion information of each second subblock.

[0314] In one possible implementation, the decoding side determines, through steps a to d below, whether the current block, the first subblock, or the second subblock satisfies the motion vector improvement condition based on the entire block. Step a: The first block determines the corresponding reference block within the reference image. The first block is currently a block, or the first subblock, or the second subblock. Step b, the M subblocks of the first block obtain the movement information of the corresponding M subblocks within the reference block, where M is a positive integer greater than 1. Step c: Classify the movement information of the M subblocks obtained and obtain P types of classification results. P is a positive integer less than or equal to M. In step d, based on the classification results for type P, it is determined whether the first block satisfies the motion vector improvement condition based on the entire block.

[0315] In this implementation method, the specific method by which the decoding side determines whether the current block, the first subblock, or the second subblock satisfies the motion vector improvement condition based on the entire block is basically the same. For the sake of simplicity, the first block is used to substitute for the current block, the first subblock, or the second subblock. Specifically, the decoding side first determines the reference block that corresponds to this first block in the reference image. Next, the first block is divided into M subblocks, and the M subblocks of the first block acquire motion information for the M corresponding subblocks in the reference block. Then, the acquired motion information for the M subblocks is classified to obtain a classification result of type P. The specific implementation process of steps a to c described above can be found in the specific explanations of S102-D1 to S102-D3 described above. By simply replacing the current block with the first block in S102-D1 to S102-D3, a classification result of type P corresponding to the first block can be obtained.

[0316] Finally, based on the classification result of type P corresponding to the first block, it is determined whether the first block satisfies the motion vector improvement condition based on the block as a whole.

[0317] For example, if P is equal to 1, it indicates that the motion information of the M subblocks in this first block is the same, and there is no need to divide the first block. Therefore, it is determined that this first block satisfies the motion vector improvement condition based on the entire block.

[0318] In another example, if P is greater than 1, we can show that the motion information of the M subblocks in this first block is not entirely the same, and the first block needs to be divided. Therefore, we determine that this first block does not satisfy the motion vector improvement condition based on the entire block.

[0319] To illustrate with an example, if, through steps a to d described above, it is determined that the current block does not satisfy the motion vector improvement condition based on the entire block, the current block is divided into multiple first subblocks based on the motion information of the reference image. Through steps a to d described above, it is determined whether each first subblock satisfies the motion vector improvement condition based on the entire block. If some first subblocks satisfy the motion vector improvement condition based on the entire block, the motion information of these partial first subblocks is improved based on the motion information of the reference image. If some first subblocks do not satisfy the motion vector improvement condition based on the entire block, each first subblock in these partial first subblocks is divided based on the motion information of the reference image to obtain multiple second subblocks. Furthermore, through steps a to d described above, it is determined whether each second subblock in these multiple second subblocks satisfies the motion vector improvement condition based on the entire block. For second subblocks that satisfy the condition, the motion information of those second subblocks is improved based on the motion information of the reference image. For second subblocks that do not satisfy the condition, the division continues, and the above steps are repeated until all subblocks satisfy the motion vector improvement condition based on the entire block. Alternatively, block division may be withheld until the size of the resulting subblocks meets a predetermined size, such as 8x8 or 4x4. By default, the motion information of these subblocks may be improved using a bidirectional optical flow motion vector improvement method based on the subblocks.

[0320] In this implementation method, the process by which the decoding side divides the current block, the first subblock, or the second subblock based on the motion information of the reference image is basically the same as in S102-D described above. For example, the decoding side first determines the reference block that corresponds to the second block in the reference image. This second block is the current block, the first subblock, or the second subblock. Next, the M subblocks of this second block acquire motion information for the M corresponding subblocks in the reference block. M is a positive integer greater than 1. Then, the acquired motion information for the M subblocks is classified to obtain P types of classification results. P is a positive integer less than or equal to M. Finally, based on the P types of classification results, the second block is divided into at least one subblock. For example, based on the subblocks that each of the P types of classification results corresponds to, the second block is divided into P subblocks. For specifics, please refer to the related explanation in S102-D described above, as it will not be repeated here.

[0321] The above-described embodiment combines Case 1 and Case 2 and explains the process by which the decoding side improves the first motion information of the current block using the motion information of the reference image, and the process of guiding block division using the motion information of the reference image.

[0322] Based on the steps described above, the decryption side obtains the second move information of the current block and then executes the following step S103.

[0323] S103, based on the second movement information, determines the predicted value of the current block.

[0324] Based on the steps described above, the decoding side considers the motion information of the reference image when improving the first motion information of the current block, effectively improving the first motion information of the current block to obtain accurate second motion information. Furthermore, when determining the predicted value of the current block based on this accurate second motion information, the prediction accuracy of the current block can be increased, further improving the video decoding performance.

[0325] In some embodiments, when the current block uses unidirectional prediction, the second motion information includes a unidirectional motion vector. Furthermore, based on this motion vector, the predicted block of the current block is determined within the reference image of the current block, and the predicted value of the current block is obtained.

[0326] In some embodiments, when the current block uses bidirectional prediction, the current block includes a first reference image and a second reference image, and the second motion information includes a first predicted direction motion vector and a second predicted direction motion vector. Thus, the decoding side determines predicted block 1 in the first reference image based on the first predicted direction motion vector in the second motion information, determines predicted block 2 in the second reference image based on the second predicted direction motion vector in the second motion information, and further obtains the predicted value of the current block based on predicted block 1 and predicted block 2. For example, the average or weighted average of predicted block 1 and predicted block 2 is determined as the predicted value of the current block.

[0327] In the video decoding method provided by the embodiment of the present invention, when decoding the current block, the decoding side first determines the first motion information of the current block, and then improves the first motion information based on the motion information of the reference image of the current block to obtain second motion information. That is, in the embodiment of the present invention, when improving the first motion information, the motion information of the reference image is taken into consideration, and the first motion information is effectively improved to obtain accurate second motion information. As a result, when determining the predicted value of the current block based on this accurate second motion information, the prediction accuracy of the current block can be increased, and the video decoding performance can be further improved.

[0328] Figures 14 to 20 are merely illustrative examples of the present application and should not be understood as limitations of the present application.

[0329] Although preferred embodiments of the present application have been described in detail above with reference to the drawings, the present application is not limited to the specific details of the embodiments described above. Within the scope of the technical idea of ​​the present application, several kinds of simple modifications can be made to the technical solution of the present application, and all of these simple modifications fall within the scope of protection of the present application. For example, each specific technical feature described in the specific embodiments described above can be combined in any appropriate manner, provided that they do not contradict each other. To avoid unnecessary redundancy, the present application does not describe the various possible combinations again. Furthermore, the various different embodiments of the present application can be combined in any way, and should be considered to be within the scope of disclosure of the present application, provided that they do not contradict the idea of ​​the present application.

[0330] Furthermore, in the various method embodiments of this application, the magnitude of the numbers of the processes described above does not indicate the order of execution, and the order of execution of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Also, in the embodiments of this application, the term "and / or" simply describes the relationship between related objects, indicating that three types of relationships may exist. Specifically, A and / or B can represent three types of situations: A exists alone, A and B exist simultaneously, or B exists alone. Also, the letter " / " in this application generally indicates that the preceding and succeeding related objects are in an "or" relationship.

[0331] The method embodiments of the present application have been described in detail above with reference to Figures 14 to 20. Hereinafter, the apparatus embodiments of the present application will be described in detail with reference to Figure 21.

[0332] Figure 21 is a schematic block diagram of a video decoding device according to one embodiment of the present invention. This video decoding device 10 is applied to the video decoder described above.

[0333] As shown in Figure 21, the video decoding device 10 is Currently, a decision unit 11 for determining the first movement information of the block, An improvement unit 12 for improving the first motion information and obtaining second motion information for the current block based on the motion information of the reference image of the current block, The system includes a prediction unit 13 for determining the predicted value of the current block based on the second motion information.

[0334] In some embodiments, the improvement unit 12 is used to determine a reference block corresponding to the current block in the reference image based on the first motion information, to determine the time-domain motion information of the current block as third motion information based on the motion information of the reference block, to improve the first motion information based on the third motion information, and to obtain second motion information.

[0335] In some embodiments, the improvement unit 12 is specifically used to determine a fourth motion information based on the third motion information and the first motion information, and to determine the second motion information based on the fourth motion information.

[0336] In some embodiments, the improvement unit 12 is specifically used to determine the average value of the third motion information and the first motion information as the fourth motion information.

[0337] In some embodiments, the improvement unit 12 is used to determine weights corresponding to the third motion information and the first motion information, to determine a weighted average of the third motion information and the first motion information based on the weights, and to determine the weighted average as the fourth motion information.

[0338] In some embodiments, the weight of the first motion information is greater than the weight of the third motion information.

[0339] In some embodiments, the improvement unit 12 is specifically used to obtain the second motion information by searching within the reference image, using the corresponding position of the fourth motion information in the reference image as the search center point for the second motion information.

[0340] In some embodiments, the reference image includes a first reference image and a second reference image, and both the second motion information and the fourth motion information include first predicted direction motion information and second predicted direction motion information. Specifically, the improvement unit 12 uses the corresponding position of the first predicted direction motion information in the first reference image in the fourth motion information as the search center point for the first predicted direction motion information in the second motion information, and the corresponding position of the second predicted direction motion information in the second reference image in the fourth motion information as the search center point for the second predicted direction motion information in the second motion information, performs motion information retrieval within a predetermined search range of the first and second reference images, determines the bidirectional matching cost of each pair of bidirectional motion information found (each pair of bidirectional motion information includes one first predicted direction motion information and one second predicted direction motion information), and is used to determine the second motion information from the multiple pairs of bidirectional motion information found based on the bidirectional matching cost.

[0341] In some embodiments, the improvement unit 12 is used to determine a first predicted block in the first reference image based on a first predicted directional motion information in the i-th pair of bidirectional motion information, to determine a second predicted block in the second reference image based on a second predicted directional motion information in the i-th pair of bidirectional motion information (where i is a positive integer), to determine the matching cost of the first and second predicted blocks, and to determine the bidirectional matching cost of the i-th pair of bidirectional motion information based on the matching cost of the first and second predicted blocks.

[0342] In some embodiments, the improvement unit 12 is specifically used to determine, as the second motion information, the pair of bidirectional motion information with the minimum bidirectional matching cost among the multiple pairs of bidirectional motion information that have been searched.

[0343] In some embodiments, the improvement unit 12 is used to determine the second motion information by using the corresponding position of the first motion information in the reference image as the search center point for the second motion information, performing a motion information search within the reference image, determining a first cost for each candidate motion information found, determining a cost coefficient corresponding to the candidate motion information based on the candidate motion information and the fourth motion information, modifying the first cost based on the cost coefficient corresponding to the candidate motion information to obtain a second cost for the candidate motion information, and determining the second motion information based on the second costs of the multiple candidate motion information found.

[0344] In some embodiments, the reference image includes a first reference image and a second reference image, and the first motion information, the second motion information, the fourth motion information, and the candidate motion information all include first predicted direction motion information and second predicted direction motion information. Specifically, the improvement unit 12 uses the corresponding position of the first predicted direction motion information in the first reference image in the first motion information as the search center point for the first predicted direction motion information in the second motion information, and the corresponding position of the second predicted direction motion information in the second reference image in the first motion information as the search center point for the second predicted direction motion information in the second motion information, and performs motion information retrieval within a predetermined search range of the first and second reference images, and is used to determine the first cost of each searched candidate motion information.

[0345] In some embodiments, if the cost coefficient corresponding to the candidate motion information includes a first cost coefficient and a second cost coefficient, the improvement unit 12 is used specifically to determine a first cost coefficient corresponding to the first predicted direction motion information in the candidate motion information based on the first predicted direction motion information in the candidate motion information and the first predicted direction motion information in the fourth motion information, and to determine a second cost coefficient corresponding to the second predicted direction motion information in the candidate motion information based on the second predicted direction motion information in the candidate motion information and the second predicted direction motion information in the fourth motion information.

[0346] In some embodiments, the improvement unit 12 is used specifically to determine the absolute value of the difference between the i-th predicted direction motion information in the candidate motion information and the i-th predicted direction motion information in the fourth motion information (where i is one or two), and to determine the i-cost coefficient based on the absolute value of the difference (where the i-cost coefficient has a negative correlation with the absolute value of the difference).

[0347] In some embodiments, the improvement unit 12 is specifically used to determine the minimum of the absolute value of the difference and a first predetermined value, and to determine the i-cost coefficient based on the minimum value.

[0348] In some embodiments, the improvement unit 12 is specifically used to determine the sum of the minimum value and the second predetermined value as the i-th cost coefficient.

[0349] In some embodiments, the improvement unit 12 is specifically used to modify the first cost of the candidate motion information and obtain the second cost of the candidate motion information based on the first cost coefficient and the second cost coefficient.

[0350] In some embodiments, the improvement unit 12 is specifically used to multiply the first cost by the first cost coefficient and the second cost coefficient to obtain the second cost of the candidate motion information.

[0351] In some embodiments, the improvement unit 12 is specifically used to determine the candidate motion information with the minimum second cost among the searched candidate motion information as the second motion information.

[0352] In some embodiments, the improvement unit 12 improves the first motion information based on the third motion information and, before obtaining the second motion information, further determines the difference value between the first motion information and the third motion information. If the difference value is less than or equal to a predetermined threshold, it improves the first motion information based on the third motion information and obtains the second motion information.

[0353] In some embodiments, the improvement unit 12 is used to divide the current block into at least one subblock based on the motion information of the reference image, improve the first motion information of the i-th subblock of the at least one subblock to obtain second motion information of the i-th subblock (where i is a positive integer), and obtain second motion information of the current block based on the second motion information of the N subblocks.

[0354] In some embodiments, the improvement unit 12 is used to determine a reference block corresponding to the i-th subblock in the reference image based on the first motion information of the i-th subblock, to determine a third motion information of the i-th subblock moving from the current image to the reference image according to the motion information of the reference block of the i-th subblock, to improve the first motion information of the i-th subblock based on the third motion information, and to obtain a second motion information of the i-th subblock.

[0355] In some embodiments, the improvement unit 12 is specifically used to improve the first motion information N times based on the motion information of the reference image of the current block to obtain the second motion information (where N is a positive integer greater than 1).

[0356] In some embodiments, the improvement unit 12 specifically improves the motion information of each subblock among at least one subblock corresponding to the j-th time, obtains improved motion information of at least one subblock corresponding to the j-th time, if j is equal to 1, the subblock corresponding to the j-th time is the current block, and based on the motion information of the reference image, it divides each subblock among at least one subblock corresponding to the j-th time into blocks to obtain at least one subblock corresponding to the j+1-th time, improves the motion information of each subblock among at least one subblock corresponding to the j+1-th time, repeats this process N times, and is used to obtain the second motion information.

[0357] In some embodiments, the improvement unit 12 is used to improve the first motion information based on the motion information of the reference image of the current block, and before obtaining the second motion information of the current block, it further determines whether the current block satisfies a predetermined motion vector improvement condition based on the entire block. If the current block satisfies the motion vector improvement condition based on the entire block, it is used to improve the first motion information based on the motion information of the reference image of the current block and obtain the second motion information of the current block.

[0358] In some embodiments, if the current block does not satisfy the motion vector improvement conditions based on the entire block, the improvement unit 12 further divides the current block based on the motion information of the reference image to obtain a plurality of first subblocks, determines for any of the plurality of first subblocks whether the first subblock satisfies the motion vector improvement conditions based on the entire block, and if the first subblock does not satisfy the motion vector improvement conditions based on the entire block, divides the first subblock based on the motion information of the reference image to obtain a plurality of second subblocks, determines for any of the plurality of second subblocks whether the second subblock satisfies the motion vector improvement conditions based on the entire block, and continues this process until the divided subblocks satisfy the motion vector improvement conditions based on the entire block or the size of the divided subblocks satisfies a predetermined size.

[0359] In some embodiments, the improvement unit 12 specifically determines the corresponding reference block in the reference image where the first block is the current block, the first subblock, or the second subblock; the M subblocks of the first block acquire motion information of the corresponding M subblocks in the reference block (where M is a positive integer greater than 1); classifies the acquired motion information of the M subblocks to obtain P types of classification results (where P is a positive integer less than or equal to M); and uses the P types of classification results to determine whether the first block satisfies the motion vector improvement conditions based on the entire block.

[0360] In some embodiments, the improvement unit 12 is used specifically to determine that the first block satisfies the motion vector improvement condition based on the entire block when P is equal to 1, and to determine that the first block does not satisfy the motion vector improvement condition based on the entire block when P is greater than 1.

[0361] In some embodiments, the improvement unit 12 is used to divide the second block into at least one subblock based on the following: the second block determines the corresponding reference block in the reference image (the second block is any one of the current block, the first subblock, the second subblock, and the subblock corresponding to the jth time); the M subblocks of the second block acquire motion information of the M corresponding subblocks in the reference block (where M is a positive integer greater than 1); classify the acquired motion information of the M subblocks to obtain P types of classification results (where P is a positive integer less than or equal to M); and divide the second block into at least one subblock based on the P types of classification results.

[0362] In some embodiments, the improvement unit 12 is specifically used to divide the second block into P subblocks based on the P classification results, each corresponding to a subblock.

[0363] The apparatus embodiments and method embodiments are interchangeable, and similar descriptions should be found in the method embodiments. To avoid duplication, they will not be repeated here. Specifically, the apparatus 10 shown in Figure 21 can perform the decoding method on the decoding side of the embodiment of the present application, and the aforementioned and other operations and / or functions of each unit in the apparatus 10 are for realizing the corresponding processes in each method, such as the decoding method on the decoding side described above. For brevity, they will not be repeated here.

[0364] The above description, with reference to the drawings, describes the apparatus and system according to the embodiment of the present application from the perspective of functional units. It is understood that the functional units may be implemented in hardware form, in software form by instructions, or in combination of hardware and software units. Specifically, each step of the embodiment of the method in the embodiment of the present application can be performed by hardware integrated logic circuits and / or software form instructions in a processor. The steps of the method disclosed in the embodiment of the present application may be completed by being performed directly by a hardware decode processor, or by being completed by a combination of hardware and software units in a decode processor. Optionally, the software units may be located in art-mature storage media such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable read-only memory, and registers. The storage media resides in memory, and the processor reads information from memory and combines it with its own hardware to complete the steps of the embodiment of the method described above.

[0365] Figure 22 is a schematic block diagram of an electronic device according to an embodiment of the present application.

[0366] As shown in Figure 22, the electronic device 30 may be a video decoder as described in the embodiment of the present application, and the electronic device 30 may include a memory 33 and a processor 32. The memory 33 is used to store the computer program 34 and to transmit the program code 34 to the processor 32. In other words, the processor 32 can realize the method in the embodiment of the present invention by calling and executing the computer program 34 from the memory 33.

[0367] For example, the processor 32 is used to execute the steps in the method 200 described above based on instructions in the computer program 34.

[0368] In some embodiments of the present application, the processor 32 is This includes, but is not limited to, general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0369] In some embodiments of the present application, the memory 33 includes, but is not limited to, volatile memory and / or non-volatile memory. Here, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), or flash memory. Volatile memory is random access memory (RAM), which functions as an external high-speed cache. Many forms of RAM are available, for example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synch-link dynamic random access memory (SLDRAM), and direct Rambus random access memory (DR RAM).

[0370] In some embodiments of the present application, the computer program 34 may be divided into one or more units. The one or more units are stored in the memory 33 and executed by the processor 32 to complete the method provided by the present application. The one or more units are a set of computer program instruction segments capable of performing a specific function, and the instruction segments are used to describe the process by which the computer program 34 is executed in the electronic device 30.

[0371] As shown in Figure 22, the electronic device 30 may further include a transceiver 33. The transceiver 33 can be connected to the processor 32 or the memory 33.

[0372] Here, the processor 32 can control the transceiver 31 to communicate with other devices, specifically by transmitting information or data to other devices or receiving information or data transmitted from other devices. The transceiver 31 may include a transmitter and a receiver. The transceiver 31 may further include an antenna, and the number of antennas may be one or more.

[0373] Each component of the electronic device 30 is connected via a bus system. This bus system includes a data bus, a power bus, a control bus, and a status signal bus.

[0374] The present invention further provides a computer storage medium. The computer storage medium stores a computer program, and when the computer program is executed by a computer, the computer is made to execute the method according to the embodiment of the method described above. In other words, embodiments of the present invention further provide a computer program product including instructions. When the instructions are executed by a computer, the computer is made to execute the method according to the embodiment of the method described above.

[0375] The present invention further provides a bitstream which is generated based on the encoding method described above.

[0376] When implemented by software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and executed, a flow or function relating to the embodiments of the present application is generated, in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted by wired means (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless means (e.g., infrared, radio, microwave, etc.) from one website, computer, server, or data center to another. The computer-readable storage medium may be any available medium accessible to the computer, or a data storage device such as a server or data center that integrates one or more available media. The usable media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid state drives (SSDs)), etc.

[0377] As those skilled in the art will understand, each example unit and algorithmic step described in the embodiments of this application may be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the invention. Skilled technicians may implement the described functions using different methods for each specific application, but such implementation will not be considered beyond the scope of this application.

[0378] In some embodiments provided herein, it should be understood that the disclosed systems, apparatus, and methods may be implemented in other ways. For example, the embodiments of the apparatus described above are merely schematic. For example, the division of the units is merely a division of logical functions, and other division methods may exist in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Also, the mutual coupling, direct coupling, or communication connection indicated or discussed may be via some interface. Indirect coupling or communication connection between apparatus or units can be implemented in electrical, mechanical or other forms.

[0379] Units described as separate components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Depending on the actual needs, some or all of the units can be selected to achieve the objectives of this embodiment. For example, each functional unit in each embodiment of this application may be integrated into a single processing unit, each unit may exist individually in physical form, or two or more units may be integrated into a single unit.

[0380] The above describes only the specific implementation of the present application, but the scope of protection is not limited thereto. Any modification or substitution that a person skilled in the art can easily conceive within the technical scope disclosed in this application should be included in the scope of protection. Therefore, the scope of protection of this application shall be equivalent to the scope of protection of the claims.

Claims

1. A video decoding method, Currently, the first step is to determine the block's movement information, The steps include improving the first motion information based on the motion information of the reference image of the current block to obtain second motion information of the current block, The step of determining the predicted value of the current block based on the second motion information is included, A method characterized by the following:

2. The step of improving the first motion information based on the motion information of the reference image of the current block and obtaining the second motion information of the current block is: The steps include determining a reference block in the reference image that corresponds to the current block based on the first motion information, The steps include determining the time-domain motion information of the current block as third motion information based on the motion information of the reference block, The step of improving the first motion information based on the third motion information to obtain second motion information includes, The method according to feature 1.

3. The step of improving the first motion information based on the third motion information and obtaining the second motion information is: A step of determining a fourth motion information based on the third motion information and the first motion information, The step of determining the second motion information based on the fourth motion information includes, The method according to feature 2.

4. The step of determining the fourth motion information based on the third motion information and the first motion information is as follows: The process includes the step of determining the average value of the third motion information and the first motion information as the fourth motion information, The method according to feature 3.

5. The step of determining the fourth motion information based on the third motion information and the first motion information is as follows: A step of determining the weights corresponding to the third motion information and the first motion information, The steps include determining the weighted average value of the third motion information and the first motion information based on the aforementioned weights, The step includes determining the weighted average value as the fourth motion information, The method according to feature 3.

6. The weight of the first motion information is greater than the weight of the third motion information. The method according to specification 5.

7. The step of determining the second motion information based on the fourth motion information is: The step of obtaining the second motion information includes searching within the reference image, using the corresponding position of the fourth motion information in the reference image as the search center point for the second motion information, The method according to feature 3.

8. The aforementioned reference image includes a first reference image and a second reference image, and both the second motion information and the fourth motion information include a first predicted direction motion information and a second predicted direction motion information. The step of obtaining the second motion information by searching within the reference image, using the corresponding position of the fourth motion information in the reference image as the search center point for the second motion information, is as follows: A step of determining the bidirectional matching cost of each pair of bidirectional motion information found, wherein each pair of bidirectional motion information includes one first predicted direction motion information and one second predicted direction motion information, the corresponding position in the first reference image of the fourth motion information being used as the search center point for the first predicted direction motion information in the second motion information, the corresponding position in the second reference image of the second predicted direction motion information in the fourth motion information being used as the search center point for the second predicted direction motion information in the second motion information, a step of performing a motion information search within a predetermined search range of the first reference image and the second reference image, the bidirectional matching cost of each pair of bidirectional motion information being found, the bidirectional motion information of each pair including one first predicted direction motion information and one second predicted direction motion information, The process includes the step of determining the second motion information from a plurality of pairs of bidirectional motion information searched based on the aforementioned bidirectional matching cost. The method according to feature 7.

9. The step of determining the bidirectional matching cost of the bidirectional motion information for each pair explored is: A step of determining a first predicted block in the first reference image based on the first predicted direction motion information in the i-th pair of bidirectional motion information that has been searched, and determining a second predicted block in the second reference image based on the second predicted direction motion information in the i-th pair of bidirectional motion information, wherein i is a positive integer. The steps include determining the matching cost of the first prediction block and the second prediction block, The step of determining the bidirectional matching cost of the i-pair of bidirectional motion information based on the matching costs of the first prediction block and the second prediction block, The method according to feature 8.

10. The step of determining the second motion information from multiple pairs of bidirectional motion information searched based on the aforementioned bidirectional matching cost is as follows: The step includes determining the pair of bidirectional motion information with the minimum bidirectional matching cost among the multiple pairs of bidirectional motion information that have been searched as the second motion information, The method according to feature 9.

11. The step of determining the second motion information based on the fourth motion information is: The steps include: setting the corresponding position of the first motion information in the reference image as the search center point for the second motion information; performing a motion information search within the reference image; and determining the first cost of each candidate motion information found; The steps include determining a cost coefficient corresponding to the candidate motion information based on the candidate motion information and the fourth motion information, The steps include: modifying the first cost based on the cost coefficient corresponding to the candidate motion information to obtain the second cost of the candidate motion information; The steps include determining the second motion information based on the second cost of a plurality of candidate motion information that has been explored, The method according to feature 3.

12. The aforementioned reference image includes a first reference image and a second reference image, and the first motion information, the second motion information, the fourth motion information, and the candidate motion information all include first predicted direction motion information and second predicted direction motion information, and the corresponding position of the first motion information in the reference image is used as the search center point for the second motion information, and the step of performing a motion information search within the reference image and determining the first cost of each searched candidate motion information is as follows: The process includes the steps of: setting the corresponding position in the first reference image of the first predicted direction motion information in the first motion information as the search center point for the first predicted direction motion information in the second motion information; setting the corresponding position in the second reference image of the second predicted direction motion information in the first motion information as the search center point for the second predicted direction motion information in the second motion information; performing a motion information search within a predetermined search range of the first and second reference images; and determining the first cost of each candidate motion information that has been searched. The method according to 11, characterized by the features described above.

13. If the cost coefficient corresponding to the candidate movement information includes a first cost coefficient and a second cost coefficient, the step of determining the cost coefficient corresponding to the candidate movement information based on the candidate movement information and the fourth movement information is as follows: A step of determining a first cost coefficient corresponding to the first predicted direction movement information in the candidate movement information, based on the first predicted direction movement information in the candidate movement information and the first predicted direction movement information in the fourth movement information. The process includes the step of determining a second cost coefficient corresponding to the second predicted direction movement information in the candidate movement information, based on the second predicted direction movement information in the candidate movement information and the second predicted direction movement information in the fourth movement information. The method according to 12, characterized by the features described above.

14. The step of determining the i-th cost coefficient corresponding to the i-th predicted direction movement information in the candidate movement information, based on the i-th predicted direction movement information in the candidate movement information and the i-th predicted direction movement information in the fourth movement information, is: A step of determining the absolute value of the difference between the i-th predicted direction motion information in the candidate motion information and the i-th predicted direction motion information in the fourth motion information, wherein i is one or two. The steps include determining the i-cost coefficient based on the absolute value of the difference, wherein the i-cost coefficient has a negative correlation with the absolute value of the difference. The method according to the present invention, characterized by the present invention.

15. The step of determining the i-cost coefficient based on the absolute value of the difference is as follows: The steps include determining the minimum value among the absolute value of the difference and the first predetermined value, The step of determining the i-cost coefficient based on the minimum value is included, The method according to feature 14.

16. The step of determining the i-cost coefficient based on the minimum value is: The step includes determining the sum of the minimum value and the second predetermined value as the i-cost coefficient, The method according to the present invention, characterized by the present invention.

17. The step of modifying the first cost based on the cost coefficient corresponding to the candidate motion information to obtain the second cost of the candidate motion information is: The process includes the step of modifying the first cost of the candidate motion information based on the first cost coefficient and the second cost coefficient to obtain the second cost of the candidate motion information. The method according to the present invention, characterized by the present invention.

18. The step of modifying the first cost of the candidate movement information and obtaining the second cost of the candidate movement information based on the first cost coefficient and the second cost coefficient is: The process includes the step of multiplying the first cost by the first cost coefficient and the second cost coefficient to obtain the second cost of the candidate motion information, The method according to feature 17.

19. The step of determining the second motion information based on the second cost of the multiple candidate motion information that has been explored is: The step includes determining the candidate motion information with the minimum second cost among the multiple candidate motion information searched as the second motion information, The method according to 12, characterized by the features described above.

20. Before improving the first motion information based on the third motion information and obtaining the second motion information, the method is: The step includes determining the difference value between the first motion information and the third motion information, The step of improving the first motion information based on the third motion information and obtaining the second motion information is: If the difference value is less than or equal to a predetermined threshold, the process includes the step of improving the first motion information based on the third motion information to obtain the second motion information. The method according to any one of features 2 to 19.

21. The step of improving the first motion information based on the motion information of the reference image of the current block and obtaining the second motion information of the current block is: The steps include dividing the current block into at least one subblock based on the motion information of the reference image, A step of improving the first motion information of the i-th subblock among the at least one subblock and obtaining the second motion information of the i-th subblock, wherein i is a positive integer; The process includes the step of obtaining second movement information for the current block based on second movement information for the N subblocks, The method according to any one of 1 to 19, characterized by the features described herein.

22. The step of improving the first motion information of the i-th subblock and obtaining the second motion information of the i-th subblock is: The steps include determining a reference block in the reference image that corresponds to the i-th subblock based on the first motion information of the i-th subblock, The steps include determining a third motion information that the i-th subblock moves from the current image to the reference image according to the motion information of the reference block of the i-th subblock, The process includes the step of improving the first motion information of the i-th subblock based on the third motion information to obtain the second motion information of the i-th subblock, The method according to feature 21.

23. The step of improving the first motion information based on the motion information of the reference image of the current block and obtaining the second motion information of the current block is: A step of improving the first motion information N times based on the motion information of the reference image of the current block to obtain second motion information, wherein N is a positive integer greater than 1. The method according to any one of 1 to 19, characterized by the features described herein.

24. The step of improving the first motion information N times based on the motion information of the reference image of the current block to obtain the second motion information is: A step of improving the motion information of each subblock among at least one subblock corresponding to the j-th time, and obtaining improved motion information of at least one subblock corresponding to the j-th time, wherein if j is equal to 1, the subblock corresponding to the j-th time is the current block. Based on the motion information of the reference image, the steps include dividing each subblock of the at least one subblock corresponding to the j-th time into blocks to obtain at least one subblock corresponding to the (j+1)th time, The process includes the step of improving the motion information of each subblock among at least one subblock corresponding to the j+1th time, repeating this process N times, and obtaining the second motion information. The method according to the feature of 23.

25. Before improving the first motion information based on the motion information of the reference image of the current block and obtaining the second motion information of the current block, the method is: The process further includes determining whether the current block satisfies the motion vector improvement condition based on a predetermined block as a whole, The step of improving the first motion information based on the motion information of the reference image of the current block and obtaining the second motion information of the current block is: If the current block satisfies the motion vector improvement conditions based on the entire block, the process includes the step of improving the first motion information based on the motion information of the reference image of the current block to obtain second motion information of the current block. The method according to any one of 1 to 19, characterized by the features described herein.

26. If the current block does not satisfy the motion vector improvement condition based on the entire block, the method Based on the motion information of the aforementioned reference image, the current block is divided into blocks to obtain a plurality of first subblocks. A step of determining whether any of the plurality of first subblocks satisfies the motion vector improvement condition based on the entire block, If the first subblock does not satisfy the motion vector improvement conditions based on the entire block, the first subblock is divided into blocks based on the motion information of the reference image to obtain a plurality of second subblocks. The process further includes the step of determining, for any of the plurality of second subblocks, whether the second subblock satisfies the motion vector improvement condition based on the entire block, repeating this process until the divided subblock satisfies the motion vector improvement condition based on the entire block, or until the size of the divided subblock satisfies a predetermined size. The method according to the present invention of the present invention.

27. Determining whether the first block satisfies the motion vector improvement condition based on the entire block is: A step of determining the first block to be a corresponding reference block in the reference image, wherein the first block is the current block, the first subblock, or the second subblock, A step in which M subblocks of the first block obtain movement information of corresponding M subblocks within the reference block, wherein M is a positive integer greater than 1; A step of classifying the movement information of the M subblocks obtained and obtaining P types of classification results, wherein P is a positive integer less than or equal to M, The process includes the step of determining whether the first block satisfies the motion vector improvement conditions based on the entire block, based on the classification results of type P. The method according to the present invention of the present invention.

28. The step of determining whether the first block satisfies the motion vector improvement condition based on the entire block, based on the classification result of type P, is: If P is equal to 1, the first block satisfies the motion vector improvement condition based on the entire block. The step of determining that if P is greater than 1, the first block does not satisfy the motion vector improvement condition based on the entire block, The method according to feature 27.

29. Based on the motion information of the aforementioned reference image, the second block is divided into blocks as follows: The step of determining the second block to be a corresponding reference block in the reference image, wherein the second block is any one of the current block, the first subblock, the second subblock, and the subblock corresponding to the jth time, A step in which M subblocks of the second block obtain movement information of corresponding M subblocks within the reference block, wherein M is a positive integer greater than 1. A step of classifying the movement information of the M subblocks obtained and obtaining P types of classification results, wherein P is a positive integer less than or equal to M, The step of dividing the second block into at least one subblock based on the classification result of type P, The method according to 21, 24, or 26, characterized by the features described above.

30. The step of dividing the second block into at least one subblock based on the classification result of type P is as follows: The step includes dividing the second block into P subblocks based on the corresponding subblocks for each of the P classification results, The method according to feature 29.

31. Currently, a decision unit for determining the first movement information of the block, An improvement unit for improving the first motion information based on the motion information of the reference image of the current block and obtaining the second motion information of the current block, A prediction unit for determining the predicted value of the current block based on the second motion information, A video decoding device characterized by the following features.

32. Electronic equipment including a processor and memory, The aforementioned memory is used to store computer programs. The aforementioned processor is used to perform the methods described in claims 1 to 30 by calling and executing a computer program stored in the memory. electronic equipment.

33. A computer-readable storage medium for storing computer programs, The computer program causes the computer to execute the methods described in claims 1 to 30. Computer-readable storage medium.

Images