Video encoding method and device, electronic device, storage medium, program product and method for generating bit stream

Abstract

The present disclosure relates to a video encoding method and its device, electronic device, storage medium, program product and method for generating a bitstream, the method comprising: obtaining a motion vector of a co-located block corresponding to a current block in an adjacent frame of the current frame; deriving a temporal motion vector candidate of the current block based on the motion vector of the co-located block; adding the temporal motion vector candidate of the current block to a motion vector candidate list of the current block; encoding the current block based on the motion vector candidate list of the current block. In this way, in the video encoding pre-analysis, by additionally adding a temporal motion vector candidate for characterizing the temporal motion consistency of the video frame in the process of constructing the motion vector candidate, a large prediction error can be effectively avoided in the coding unit at the same position on adjacent frames, that is, the degree of motion discontinuity between consecutive frames can be effectively reduced, thereby improving the accuracy of the pre-analysis, thereby improving the effect of encoding parameter optimization, and thereby improving the video compression efficiency and video quality.

Description

Technical Field

[0001] The present application relates to video coding and compression, and more specifically, to a video coding method and apparatus thereof, an electronic device, a storage medium, a program product, and a method for generating a bitstream. Background Art

[0002] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send and receive or otherwise transmit digital video data through a communication network, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, before the video data is transmitted or stored, the video data can be compressed using a video codec according to one or more video codec standards. For example, video codec standards include general video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the inherent redundancy in video data. Video codecs are intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0003] Video coding pre-analysis technology aims to use a variety of pre-analysis methods to analyze the video in advance before the actual coding thread, and obtain a series of important video information including video content complexity, noise level, scene switching content, etc., so as to optimize the coding parameters of the coding thread to further improve video compression efficiency and video quality. This pre-analysis technology is adopted by many domestic and foreign video coding standards including AVC, HEVC, VVC, AVS, etc.

[0004] In order to obtain accurate inter-frame motion information of video, video coding pre-analysis technology usually adopts motion compensation prediction method similar to coding thread. However, in the motion estimation process of motion compensation prediction in related technologies, there will be large prediction deviation between adjacent frame coding units, which will affect the accuracy of coding pre-analysis and thus reduce the coding quality. Summary of the invention

[0005] Examples of the present disclosure provide a video encoding method and apparatus thereof, an electronic device, a storage medium, a program product, and a method for generating a bitstream.

[0006] According to a first aspect of the present disclosure, a video encoding method is provided, comprising: obtaining a motion vector of a co-located block corresponding to a current block in an adjacent frame of the current frame, wherein the adjacent frame is an encoded frame; deriving a temporal motion vector candidate of the current block based on the motion vector of the co-located block; adding the temporal motion vector candidate of the current block to a motion vector candidate list of the current block; and encoding the current block based on the motion vector candidate list of the current block.

[0007] Optionally, the video encoding method also includes: taking the current frame as a reference, searching for the encoded frame closest to the current frame in the N forward frames and N backward frames of the current frame in order from near to far as the adjacent frame, wherein N is a preset query length.

[0008] Optionally, deriving the temporal motion vector candidate of the current block based on the motion vector of the co-located block includes: acquiring a first distance and a second distance, wherein the first distance indicates the distance between the current frame and a reference frame of the current frame, and the second distance indicates the distance between the adjacent frame and the reference frame of the adjacent frame; and deriving the temporal motion vector candidate of the current block based on the first distance, the second distance and the motion vector of the co-located block.

[0009] Optionally, deriving the time domain motion vector candidate of the current block based on the first distance, the second distance and the motion vector of the co-located block includes: calculating the product of the ratio between the first distance and the second distance and the motion vector of the co-located block as the time domain motion vector candidate of the current block.

[0010] Optionally, adding the time-domain motion vector candidate of the current block to the motion vector candidate list of the current block includes: determining whether the time-domain motion vector candidate of the current block is repeated with an existing motion vector candidate in the motion vector candidate list of the current block; if the time-domain motion vector candidate of the current block is not repeated with the existing motion vector candidate, adding the time-domain motion vector candidate of the current block to the motion vector candidate list of the current block; otherwise, not adding the time-domain motion vector candidate of the current block to the motion vector candidate list of the current block.

[0011] Optionally, adding the time domain motion vector candidate of the current block to the motion vector candidate list of the current block includes: calculating the similarity between the time domain motion vector candidate and the existing motion vector candidates in the motion vector candidate list of the current block; if the similarity is less than a preset threshold, adding the time domain motion vector candidate of the current block to the motion vector candidate list of the current block; otherwise, not adding the time domain motion vector candidate of the current block to the motion vector candidate list of the current block.

[0012] According to a second aspect of the present disclosure, a video encoding device is provided, comprising: a motion vector acquisition module, configured to acquire a motion vector of a co-located block corresponding to a current block in an adjacent frame of the current frame, wherein the adjacent frame is an encoded frame; a time domain motion vector derivation module, configured to derive a time domain motion vector candidate of the current block based on the motion vector of the co-located block; a motion vector adding module, configured to add the time domain motion vector candidate of the current block to a motion vector candidate list of the current block; and an encoding module, configured to encode the current block based on the motion vector candidate list of the current block.

[0013] Optionally, the video encoding device also includes: a neighboring frame search module, which is configured to use the current frame as a reference and search for the encoded frame closest to the current frame in the N forward frames and N backward frames of the current frame in order from near to far as the neighboring frame, wherein N is a preset query length.

[0014] Optionally, the time domain motion vector derivation module is configured to: obtain a first distance and a second distance, wherein the first distance indicates the distance between the current frame and a reference frame of the current frame, and the second distance indicates the distance between the adjacent frame and the reference frame of the adjacent frame; and derive a time domain motion vector candidate of the current block based on the first distance, the second distance and the motion vector of the co-located block.

[0015] Optionally, the temporal motion vector derivation module is configured to: calculate a product of a ratio between the first distance and the second distance and a motion vector of the co-located block as a temporal motion vector candidate of the current block.

[0016] Optionally, the motion vector adding module is configured to: determine whether the time domain motion vector candidate of the current block is repeated with an existing motion vector candidate in a motion vector candidate list of the current block; if the time domain motion vector candidate of the current block is not repeated with the existing motion vector candidate, add the time domain motion vector candidate of the current block to the motion vector candidate list of the current block; otherwise, do not add the time domain motion vector candidate of the current block to the motion vector candidate list of the current block.

[0017] Optionally, the motion vector adding module is configured to: calculate the similarity between the time domain motion vector candidate and the existing motion vector candidates in the motion vector candidate list of the current block; if the similarity is less than a preset threshold, add the time domain motion vector candidate of the current block to the motion vector candidate list of the current block; otherwise, do not add the time domain motion vector candidate of the current block to the motion vector candidate list of the current block.

[0018] According to a third aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a memory, coupled to the one or more processors and configured to store instructions that can be executed by the one or more processors, wherein the one or more processors are configured to execute the video encoding method according to the present disclosure when executing the instructions.

[0019] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium for storing computer-executable instructions is provided, wherein the computer-executable instructions, when executed by one or more computer processors, cause the one or more computer processors to perform the video encoding method according to the present disclosure.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream comprises: video data generated by the video encoding method according to the present disclosure.

[0021] According to a sixth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the video encoding method according to the present disclosure when executed by at least one processor.

[0022] According to a seventh aspect of the present disclosure, a method for generating a bit stream is provided, comprising: generating a bit stream according to the video encoding method of the present disclosure.

[0023] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0024] In the present disclosure, in the video coding pre-analysis, by additionally adding time domain motion vector candidates for characterizing the temporal motion consistency of video frames in the motion vector candidate construction process, large prediction errors in coding units at the same position on adjacent frames can be effectively avoided, that is, the degree of motion discontinuity between consecutive frames can be effectively reduced, thereby improving the accuracy of pre-analysis, thereby improving the coding parameter optimization effect, and thereby improving video compression efficiency and video quality.

[0025] It is to be understood that both the foregoing general description and the following detailed description are examples only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate examples according to the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0027] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0028] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0029] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0030] Figure 4 It is a schematic diagram showing the implementation process of pre-analysis motion compensation prediction in the related art.

[0031] Figure 5 It is a schematic diagram showing the implementation process of "motion vector candidate construction" and "motion vector search" in the related art.

[0032] Figure 6 is a schematic diagram showing the cause of time domain deviation in the related art.

[0033] Figure 7 is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0034] Figure 8 FIG. 4 is a schematic diagram showing a process of searching for an adjacent frame (co-located frame) according to an exemplary embodiment of the present disclosure.

[0035] Fig. 9 is a schematic diagram illustrating an encoding order of each of 32 video frames according to an exemplary embodiment of the present disclosure.

[0036] Fig.10is a schematic diagram illustrating a temporal motion vector candidate construction process according to an exemplary embodiment of the present disclosure.

[0037] Fig.11 is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment of the present disclosure.

[0038] Fig.12 A schematic diagram of a computing environment coupled to a user interface is shown. DETAILED DESCRIPTION

[0039] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the drawings are used to distinguish objects, but not to describe any specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0041] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0042] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0043] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0044] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0045] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0046] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0047] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0048] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0049] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0050] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0051] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0052] like Figure 2 As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0053] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0054] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0055] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0056] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0057] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0058] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0059] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0060] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0061] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0062] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0063] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0064] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0065] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0066] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0067] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0068] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0069] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0070] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0071] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0072] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0073] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0074] As analyzed above, video coding pre-analysis technology usually adopts a motion compensation prediction method similar to the encoding thread. Figure 4 is a schematic diagram showing the implementation process of pre-analysis motion compensation prediction in the related art. Figure 4 The motion compensation prediction process of video coding pre-analysis technology can mainly include three parts: "coding unit division", "motion estimation" and "motion compensation". Specifically:

[0075] First, the current pre-analysis frame can be divided into multiple non-overlapping blocks of 8×8 size. Then, motion estimation can be performed on each block based on the reverse scanning order to obtain the motion vector between it and the reference frame (the encoded adjacent frame), wherein the motion estimation process can mainly include two parts: "motion vector candidate construction" and "motion vector search". Next, motion compensation can be performed based on the acquired motion vector, and the sum of absolute transformed differences (SATD) between the motion compensated matching block and the current block can be calculated. Then, the block with the smallest SATD on multiple reference frames can be used as the final matching block of the current block. Through the pre-analysis motion compensation prediction process, the encoder can obtain motion information that accurately describes the video content, which can provide an important reference for subsequent encoding parameter optimization.

[0076] The above-mentioned "motion estimation" process and "motion compensation" process are described in detail below.

[0077] Motion Estimation

[0078] As mentioned above, the "motion estimation" process can include two parts: "motion vector candidate construction" and "motion vector search". In the "motion vector candidate construction" part, the current block can use the motion information of the surrounding coded units in the spatial domain to construct an initial motion vector candidate list, and can obtain the optimal motion vector MVP after traversing all motion vector candidates through motion compensation prediction and SATD-based matching criteria.

[0079] Figure 5 2 is a schematic diagram showing the implementation process of "motion vector candidate construction" and "motion vector search" in the related art. Figure 5 , the current block can fill the motion vectors of the coded units (coded blocks) located at its right position, bottom position, bottom left position and bottom right position into the motion vector candidate list of the current block in order. Then, each motion vector candidate in the motion vector candidate list of the current block can be traversed, and the SATD between the pixel block pointed to by each motion vector candidate and the current block can be calculated. Next, the motion vector candidate with the smallest SATD can be used as the optimal motion vector MVP of the current block. Finally, a variable radius hexagonal search algorithm or a hexagonal search algorithm can be used to search near the optimal motion vector MVP to obtain a better motion vector.

[0080] Motion Compensation

[0081] When performing motion compensation for the current block, the corresponding pixel value can be obtained from the reference frame based on the optimal motion vector MVP of the current block that has been searched. Furthermore, if the optimal motion vector MVP is an integer, the pixel value of the corresponding position can be directly extracted; if the optimal motion vector MVP is a sub-pixel, it is necessary to use a predetermined tap filter coefficient to interpolate the reference block in the horizontal and vertical directions, respectively, and then the final motion compensation block can be obtained. In addition, in the case of bidirectional prediction, the encoder can perform motion compensation (Motion Compensation, MC) on the reference frames in both directions, that is, MC can be performed on the forward reference frame and the backward reference frame respectively. Then, the results of the prediction blocks in the forward and backward directions can be averaged and summed to improve the prediction accuracy.

[0082] It should be noted that the above Figure 5 In the construction of motion vector candidates and motion vector search for video coding pre-analysis, the spatial motion consistency of the coding unit is mainly used, where "spatial motion consistency" refers to "spatial periphery consistency", and "spatial periphery" means that the referenced coding block is a coded block that is on the same video frame as the current block and is located near the current block. However, since video content often has a high degree of temporal motion consistency, if only spatial consistency is considered, it may lead to a large prediction deviation in adjacent frame coding units, where "temporal motion consistency" means that the motion of objects in the same block is coherent and stable. In addition, an image block that is located on an adjacent frame and is consistent with the current block in the temporal domain can be called a "co-located block", that is, the spatial position of the "co-located block" on the adjacent frame of the current frame is the same as the spatial position of the current block on the current frame.

[0083] Figure 6 Schematic diagram showing the causes of time domain deviation in the related art. Figure 6 In the process of motion vector candidate construction and motion vector search for video coding pre-analysis, if only the reference blocks around the current block in the spatial domain are considered, and the temporal consistency between the current block and its co-located blocks in adjacent frames is ignored, it will lead to large prediction deviations in adjacent frame coding units. This deviation will affect the prediction accuracy of the coding for video content analysis, and will also cause large error fluctuations in the temporal co-located blocks, thereby reducing the coding quality.

[0084] In order to solve the above-mentioned problems existing in the related art, the video encoding method and its device, electronic device, storage medium, program product and method for generating a bitstream provided by the present invention can effectively avoid large prediction errors in coding units at the same position on adjacent frames by additionally adding time domain motion vector candidates for characterizing the time domain motion consistency of video frames in the process of motion vector candidate construction in video encoding pre-analysis, that is, it can effectively reduce the degree of motion discontinuity between consecutive frames, thereby improving the accuracy of pre-analysis, thereby improving the optimization effect of encoding parameters, and thus improving video compression efficiency and video quality.

[0085] Next, we will refer to Figures 7 to 12 The video encoding method and its device, electronic device, storage medium, program product and method for generating a bit stream in the present disclosure are introduced in detail.

[0086] Figure 7 is a flowchart illustrating a video encoding method according to an exemplary embodiment of the present disclosure.

[0087] Reference Figure 7 In step 701, a motion vector of a co-located block corresponding to the current block in an adjacent frame of the current frame may be obtained, wherein the adjacent frame may be an encoded frame. Furthermore, the "motion vector of the co-located block" may refer to the "spatial motion vector" of the co-located block.

[0088] According to an exemplary embodiment of the present disclosure, the current frame may be used as a reference, and the encoded frames closest to the current frame may be searched in the N forward frames and / or N backward frames of the current frame in order from near to far as the adjacent frames, where N is a preset query length. Exemplarily, the value of N may be, but is not limited to, 32. In addition, the above-mentioned "adjacent frame" may also be referred to as a "co-located frame".

[0089] It should be noted that, considering that adjacent video frames often have similar motion states, the present disclosure can adopt a near-to-far strategy to query the forward and backward frames adjacent to the current frame respectively, until the closest and encoded video frame to the current frame is found, and the closest and encoded video frame to the current frame can be used as the adjacent frame of the current frame, i.e., the co-located frame. Furthermore, the adjacent frames of the current frame can include the forward adjacent frames and the backward adjacent frames.

[0090] Figure 8 FIG. 2 is a schematic diagram showing a process of searching for adjacent frames (co-located frames) according to an exemplary embodiment of the present disclosure. Figure 8, the current frame is T. In the "forward" scenario, you can first query whether the (T-1)th frame has been encoded. If the (T-1)th frame has not been encoded, you can continue to query whether the (T-2)th frame has been encoded. If the (T-2)th frame has been encoded, you can set the (T-2)th frame as the forward adjacent frame (forward co-located frame) of the current frame; if the (T-2)th frame has not been encoded, you can continue to query whether the (T-3)th frame has been encoded. And so on, until the video frame closest to the current frame and has been encoded is queried. In addition, in the process of querying adjacent frames, the maximum query length N can be but is not limited to: 32. In addition, the implementation process of querying the "backward adjacent frame (backward co-located frame)" of the current frame is similar to the process of querying the forward adjacent frame (forward co-located frame), which will not be repeated here.

[0091] Next, the block in the forward co-located frame that is in the same spatial position as the current block can be used as the forward co-located block, and / or the block in the backward co-located frame that is in the same spatial position as the current block can be used as the backward co-located block. That is, in the present disclosure, at most two co-located blocks can be obtained for the current block.

[0092] In addition, if no suitable co-located block is found, the motion vector of the co-located block may not be used to derive the temporal motion vector, and a non-co-located block, for example, other blocks matching the current block may be used to derive the temporal motion vector. The present disclosure does not limit the specific method of deriving the temporal motion vector of the current block, and the aforementioned implementation is only an exemplary description.

[0093] According to an exemplary embodiment of the present disclosure, a possible video frame encoding order may be:

[0094] Assuming that the video contains a total of 32 video frames, the encoding order of these 32 video frames can be: first encode the 0th frame; then encode the 32nd frame; next, based on the encoded 0th frame and the encoded 32nd frame, encode the 16th frame. At this time, the forward reference frame of the 16th frame to be encoded is the encoded 0th frame, and the backward reference frame of the 16th frame to be encoded is the encoded 32nd frame.

[0095] Then, the 8th frame can be encoded based on the encoded 0th frame and the encoded 16th frame. At this time, the forward reference frame of the 8th frame to be encoded is the encoded 0th frame, and the backward reference frame of the 8th frame to be encoded is the encoded 16th frame.

[0096] Next, the 24th frame can be encoded based on the 16th frame that has been encoded and the 32nd frame that has been encoded. At this time, the forward reference frame of the 24th frame to be encoded is the 16th frame that has been encoded, and the backward reference frame of the 24th frame to be encoded is the 32nd frame that has been encoded. By analogy, the encoding of 32 frames can be completed in sequence.

[0097] Fig. 9 is a schematic diagram showing the encoding order of each of the 32 video frames according to an exemplary embodiment of the present disclosure. Fig. 9 , the numbers in the first row: 0 to 32 represent the 0th to 32nd frames, and the numbers under each video frame represent the encoding order of the corresponding video frame.

[0098] Take the 6th frame as the current frame as an example, refer to Fig. 9 , the coding order of the 6th frame is 12, the coding order of the 5th frame is 22, and the coding order of the 4th frame is 6. Therefore, the 4th frame is the forward video frame that has been encoded and is closest to the current frame (the 6th frame), so the 4th frame is the forward reference frame of the 6th frame; similarly, as mentioned above, the coding order of the 6th frame is 12, the coding order of the 7th frame is 24, and the coding order of the 8th frame is 4. Therefore, the 8th frame is the backward video frame that has been encoded and is closest to the current frame (the 6th frame), so the 8th frame is the backward reference frame of the 6th frame.

[0099] It should be noted that in the present disclosure, "forward" and "backward" are both defined based on the current frame. Specifically, a "forward video frame" is a video frame that is played before the current frame; a "backward video frame" is a video frame that is played after the current frame.

[0100] In addition, refer to Fig. 9 , since the 4th frame is the forward video frame that has been encoded and is closest to the current frame (the 6th frame), the 4th frame is also the "forward co-located frame" of the 6th frame. And, as mentioned above, the encoding order of the 4th frame is 6, the encoding order of the 3rd frame is 20, the encoding order of the 2nd frame is 10, the encoding order of the 1st frame is 18, and the encoding order of the 0th frame is 1, so the 0th frame is the forward video frame that has been encoded and is closest to the 4th frame, so the 0th frame is the forward reference frame of the forward co-located frame (the 4th frame); similarly, as mentioned above, the encoding order of the 4th frame is 6, the encoding order of the 5th frame is 22, the encoding order of the 6th frame is 12, the encoding order of the 7th frame is 24, and the encoding order of the 8th frame is 4, so the 8th frame is the backward video frame that has been encoded and is closest to the forward co-located frame (the 4th frame), so the 8th frame is the backward reference frame of the forward co-located frame (the 4th frame).

[0101] In step 702, the temporal motion vector candidate MV of the current block may be derived based on the motion vector of the co-located block. Cur .

[0102] Next, the specific implementation process of obtaining the temporal motion vector candidate of the current block is described by taking the "forward" scenario as an example:

[0103] As mentioned above, the current frame may be the 6th frame, and the co-located block on the forward co-located frame (4th frame) of the current frame (6th frame) may correspond to multiple spatial motion vectors, and these multiple spatial motion vectors may be multiple spatial motion vectors corresponding one-to-one to multiple encoded blocks adjacent to the co-located block on the forward co-located frame (4th frame) on the forward reference frame (0th frame) of the forward co-located frame. Each of the above multiple spatial motion vectors may be used to perform motion compensation on the co-located block on the forward co-located frame (4th frame), thereby obtaining multiple motion compensation blocks. Then, the SATD between each of the multiple motion compensation blocks and the co-located block on the forward co-located frame (4th frame) may be calculated. Next, the spatial motion vector corresponding to the minimum SATD among the multiple SATDs corresponding one-to-one to the multiple motion compensation blocks may be determined as the forward optimal spatial motion vector MVP of the co-located block on the forward co-located frame (4th frame). 前向-空域 Similarly, the backward optimal spatial motion vector MVP of the backward co-located block on the backward adjacent frame (backward co-located frame, frame 8) of the current frame can also be calculated. 后向-空域 , I will not go into details here.

[0104] Next, the optimal spatial motion vector MVP of the same block can be used 空域 , derive the temporal motion vector candidate MV of the current block Cur-时域 .

[0105] According to an exemplary embodiment of the present disclosure, a first distance and a second distance may also be acquired, wherein the first distance may indicate a distance between a current frame and a reference frame of the current frame, and the second distance may indicate a distance between an adjacent frame and a reference frame of the adjacent frame. Fig.10 is a schematic diagram showing a temporal motion vector candidate construction process according to an exemplary embodiment of the present disclosure. Fig.10 , you can calculate the current frame Cur and the reference frame Cur of the current frame ref The first distance between: D cur =POC cur -POC cur-ref , among which, POC cur is the playback order of the current frame (i.e., display order), POC cur-ref is the playback order of the reference frames of the current frame; it can also calculate the adjacent frame Col and the reference frame Col of the adjacent frame reThe second distance between: D col =POC col -POC col-ref , among which, POC col POC is the playback order of the same frame. col-ref is the playback order of the reference frames of the collocated frames. It should be noted that the playback order of video frames can also be called Picture Order Count (POC).

[0106] Then, based on the first distance, the second distance and the motion vector MV of the co-located block, Col , that is, the optimal spatial motion vector MVP of the aforementioned co-located block 空域 , derive the temporal motion vector candidate MV of the current block Cur-时域 In addition, the above-mentioned “first distance” and “second distance” can also be referred to as “time domain distance”.

[0107] According to an exemplary embodiment of the present disclosure, the first distance D can also be calculated as cur and the second distance D col Then the ratio between this ratio and the motion vector MV of the same block can be calculated Col The product of is used as the temporal motion vector candidate MV of the current block. Cur :

[0108] MV Cur =D cur / D col *MV Col

[0109] For example, the "forward" scenario is still used as an example. As mentioned above, the current frame is the 6th frame, that is, POC cur =6; the forward reference frame of the current frame is the 4th frame, so POC cur-ref =4; the forward adjacent frame (forward co-located frame) of the current frame is the 4th frame, then POC col =4; the forward reference frame of the forward adjacent frame (forward co-located frame) is frame 0, so POC col-ref = 0. At this time, the candidate forward temporal motion vector of the current block is:

[0110]

[0111] Among them, MV Col-前向 is the forward optimal spatial motion vector of the forward co-located block of the current block.

[0112] Similarly, the backward optimal spatial motion vector MVP of the backward co-located block on the backward adjacent frame (the backward co-located frame, the 8th frame) of the current frame can also be used. 后向-空域, calculate the backward temporal motion vector candidate MV of the current block Cur-后向时域 , I will not go into details here.

[0113] It should be noted that the temporal motion vector candidate MV of the current block is derived Cur The method is not limited to the above method, and other methods may also be used. For example, the motion vector of the co-located block may be directly used as the time domain motion vector candidate of the current block. The present disclosure does not limit the specific method of obtaining the time domain motion vector candidate of the current block. The above implementation is only an exemplary description.

[0114] In step 703, the temporal motion vector candidate of the current block may be added to the motion vector candidate list of the current block.

[0115] Exemplarily, taking the "forward" scenario as an example, the forward temporal motion vector candidate of the current block can be added to the forward spatial motion vector candidate list of the current block, wherein the forward spatial motion vector candidate list of the current block can include multiple forward spatial motion vectors, and these multiple forward spatial motion vectors can be multiple spatial motion vectors of multiple encoded blocks adjacent to the current block on the current frame (the 6th frame) that correspond one to one on the forward reference frame (the 4th frame) of the current frame.

[0116] According to an exemplary embodiment of the present disclosure, it may also be determined whether the temporal motion vector candidate of the current block is repeated with an existing motion vector candidate in the motion vector candidate list of the current block. If the temporal motion vector candidate of the current block is not repeated with the existing motion vector candidate, the temporal motion vector candidate of the current block may be added to the motion vector candidate list of the current block; otherwise, that is, if the temporal motion vector candidate of the current block is repeated with the existing motion vector candidate, the temporal motion vector candidate of the current block may not be added to the motion vector candidate list of the current block.

[0117] In this way, by determining whether the time domain motion vector candidate of the current block is repeated with the existing motion vector candidate, repeated processing of the time domain motion vector candidate that is repeated with the existing motion vector candidate can be avoided, thereby reducing unnecessary calculations and avoiding waste of computing resources.

[0118] According to an exemplary embodiment of the present disclosure, the similarity between the temporal motion vector candidate and the existing motion vector candidate in the motion vector candidate list of the current block can also be calculated. Exemplarily, the Euclidean distance between the temporal motion vector candidate (x1, y1) and the existing motion vector candidate (x2, y2) can be calculated: As the degree of similarity mentioned above.

[0119] When the above-mentioned similarity level is less than the preset threshold, it means that the time domain motion vector candidate is not similar to each existing motion vector candidate. At this time, the time domain motion vector candidate of the current block can be added to the motion vector candidate list of the current block; otherwise, that is, when the above-mentioned similarity level is greater than or equal to the preset threshold, it means that the time domain motion vector candidate is relatively similar to one or some existing motion vector candidates. At this time, the time domain motion vector candidate of the current block may not be added to the motion vector candidate list of the current block.

[0120] In step 704, the current block may be encoded based on the motion vector candidate list of the current block. Specifically, the current block may be motion compensated using the temporal motion vector candidate of the current block and each of the multiple spatial motion vectors of the current block, thereby obtaining multiple motion compensation blocks. Then, the SATD between each of the multiple motion compensation blocks and the current block may be calculated. Next, the motion vector corresponding to the minimum SATD among the multiple SATDs corresponding to the multiple motion compensation blocks may be used as the optimal motion vector of the current block.

[0121] Furthermore, the forward optimal motion vector of the current block can be obtained based on the forward temporal motion vector candidates of the current block and the forward spatial motion vector list of the current block; and / or, the backward optimal motion vector of the current block can be obtained based on the backward temporal motion vector candidates of the current block and the backward spatial motion vector list of the current block.

[0122] Next, the encoding parameters of the current frame can be obtained based on the optimal motion vector of the current block, that is, the encoding parameters of the current frame can be obtained based on the forward optimal motion vector and the backward optimal motion vector of the current block. Then, the current frame can be encoded using the obtained encoding parameters.

[0123] It should be noted that, considering that in the actual pre-analysis process, multiple threads are usually used to perform motion compensation prediction on multiple frames at the same time, in order to avoid the problem of multiple encoding inconsistencies, the video encoding method provided by the present disclosure can be executed only in the coding structure adaptive decision, MCTF and Cutree motion compensation prediction process.

[0124] Fig.11 1 is a block diagram illustrating a video encoding apparatus 1100 according to an exemplary embodiment of the present disclosure.

[0125] Reference Fig.11 The video encoding device 1100 may include a motion vector acquisition module 1101, a time domain motion vector derivation module 1102, a motion vector adding module 1103 and an encoding module 1104.

[0126] The motion vector acquisition module 1101 may acquire the motion vector of the co-located block corresponding to the current block in the adjacent frame of the current frame, wherein the adjacent frame may be an encoded frame. Moreover, the "motion vector of the co-located block" may refer to the "spatial motion vector" of the co-located block.

[0127] According to an exemplary embodiment of the present disclosure, the video encoding apparatus 1100 may further include a neighboring frame search module.

[0128] The adjacent frame search module can use the current frame as a reference, and search the N forward frames and / or N backward frames of the current frame in order from near to far for the encoded frame closest to the current frame as the adjacent frame, where N is a preset query length. Exemplarily, the value of N can be, but is not limited to, 32. In addition, the above-mentioned "adjacent frame" can also be called "co-located frame".

[0129] The temporal motion vector derivation module 1102 can derive a temporal motion vector candidate MV of the current block based on the motion vector of the co-located block. Cur .

[0130] According to an exemplary embodiment of the present disclosure, the temporal motion vector derivation module 1102 may also obtain a first distance and a second distance, wherein the first distance may indicate a distance between a current frame and a reference frame of the current frame, and the second distance may indicate a distance between a neighboring frame and a reference frame of the neighboring frame. Then, the temporal motion vector derivation module 1102 may obtain a first distance and a second distance based on the first distance, the second distance and the motion vector MV of the co-located block. Col , derive the temporal motion vector candidate MV of the current block Cur-时域 In addition, the above-mentioned “first distance” and “second distance” can also be referred to as “time domain distance”.

[0131] According to an exemplary embodiment of the present disclosure, the temporal motion vector derivation module 1102 may further calculate a ratio between the first distance and the second distance. Then, the temporal motion vector derivation module 1102 may calculate the ratio and the motion vector MV of the co-located block. Col The product of is used as the temporal motion vector candidate MV of the current block. Cur .

[0132] The motion vector adding module 1103 may add the temporal motion vector candidate of the current block to the motion vector candidate list of the current block.

[0133] According to an exemplary embodiment of the present disclosure, the motion vector adding module 1103 may also determine whether the temporal motion vector candidate of the current block is repeated with an existing motion vector candidate in the motion vector candidate list of the current block. In the case where the temporal motion vector candidate of the current block is not repeated with the existing motion vector candidate, the motion vector adding module 1103 may add the temporal motion vector candidate of the current block to the motion vector candidate list of the current block; otherwise, that is, in the case where the temporal motion vector candidate of the current block is repeated with the existing motion vector candidate, the motion vector adding module 1103 may not add the temporal motion vector candidate of the current block to the motion vector candidate list of the current block.

[0134] In this way, by determining whether the time domain motion vector candidate of the current block is repeated with the existing motion vector candidate, repeated processing of the time domain motion vector candidate that is repeated with the existing motion vector candidate can be avoided, thereby reducing unnecessary calculations and avoiding waste of computing resources.

[0135] According to an exemplary embodiment of the present disclosure, the motion vector adding module 1103 may also calculate the similarity between the temporal motion vector candidate and the existing motion vector candidates in the motion vector candidate list of the current block.

[0136] When the above-mentioned similarity level is less than the preset threshold, it means that the time domain motion vector candidate is not similar to each existing motion vector candidate. At this time, the motion vector adding module 1103 can add the time domain motion vector candidate of the current block to the motion vector candidate list of the current block; otherwise, that is, when the above-mentioned similarity level is greater than or equal to the preset threshold, it means that the time domain motion vector candidate is similar to one or some existing motion vector candidates. At this time, the motion vector adding module 1103 may not add the time domain motion vector candidate of the current block to the motion vector candidate list of the current block.

[0137] The encoding module 1104 can encode the current block based on the motion vector candidate list of the current block. Specifically, the current block can be motion compensated using the temporal motion vector candidate of the current block and each of the multiple spatial motion vectors of the current block, thereby obtaining multiple motion compensation blocks. Then, the SATD between each of the multiple motion compensation blocks and the current block can be calculated. Next, the motion vector corresponding to the minimum SATD among the multiple SATDs corresponding to the multiple motion compensation blocks can be used as the optimal motion vector of the current block.

[0138] Furthermore, the forward optimal motion vector of the current block can be obtained based on the forward temporal motion vector candidates of the current block and the forward spatial motion vector list of the current block; and / or, the backward optimal motion vector of the current block can be obtained based on the backward temporal motion vector candidates of the current block and the backward spatial motion vector list of the current block.

[0139] Next, the encoding parameters of the current frame can be obtained based on the optimal motion vector of the current block, that is, the encoding parameters of the current frame can be obtained based on the forward optimal motion vector and / or the backward optimal motion vector of the current block. Then, the current frame can be encoded using the obtained encoding parameters.

[0140] Fig.12 A computing environment 1210 is shown coupled to a user interface 1250. The computing environment 1210 may be part of a data processing server. The computing environment 1210 includes a processor 1220, a memory 1230, and an input / output (I / O) interface 1240.

[0141] The processor 1220 generally controls the overall operation of the computing environment 1210, such as operations associated with display, data acquisition, data communication, and image processing. The processor 1220 may include one or more processors for executing instructions to perform all or some of the steps in the above method. In addition, the processor 1220 may include one or more modules that facilitate the interaction between the processor 1220 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0142] The memory 1230 is configured to store various types of data to support the operation of the computing environment 1210. The memory 1230 may include predetermined software 1232. Examples of such data include instructions for any application or method operating on the computing environment 1210, video data sets, image data, etc. The memory 1230 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0143] I / O interface 1240 provides an interface between processor 1220 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 1240 may be coupled to an encoder and a decoder.

[0144] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 1230 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 1220 in a computing environment 1210 to perform the above method. In one example, the plurality of programs can be executed by a processor 1220 in a computing environment 1210 to (for example, from Figure 2 The video encoder 20 in the computing environment 1210 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1220 in the computing environment 1210 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 1220 in the computing environment 1210 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 1220 in the computing environment 1210 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0145] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0146] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 1220); and a non-transitory computer-readable storage medium or memory 1230 in which multiple programs that can be executed by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0147] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes the encoded video information generated by the above encoding method or the encoded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 1230 is also provided, wherein the plurality of programs can be executed by a processor 1220 in a computing environment 1210 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0148] In an embodiment, the computing environment 1210 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0149] In an embodiment, a method for storing a bit stream is also provided, comprising: storing the bit stream on a digital storage medium, wherein the bit stream comprises encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0150] In an embodiment, a method for transmitting a bit stream generated by the above encoder is also provided. In an embodiment, a method for receiving a bit stream to be decoded by the above decoder is also provided.

[0151] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0152] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0153] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video encoding method, characterized in that: include: Obtaining a motion vector of a co-located block corresponding to the current block in an adjacent frame of the current frame, wherein the adjacent frame is an encoded frame; Based on the motion vector of the co-located block, deriving a temporal motion vector candidate of the current block; Adding the temporal motion vector candidate of the current block to a motion vector candidate list of the current block; The current block is encoded based on the motion vector candidate list of the current block.

2. The video encoding method according to claim 1, characterized in that: The video encoding method further comprises: Taking the current frame as a reference, the encoded frames closest to the current frame are searched in N forward frames and N backward frames of the current frame in order from near to far respectively as the adjacent frames, wherein N is a preset query length.

3. The video encoding method according to claim 1, wherein: The deriving, based on the motion vector of the co-located block, a temporal motion vector candidate of the current block comprises: Acquire a first distance and a second distance, wherein the first distance indicates a distance between the current frame and a reference frame of the current frame, and the second distance indicates a distance between the adjacent frame and a reference frame of the adjacent frame; A temporal motion vector candidate of the current block is derived based on the first distance, the second distance, and the motion vector of the co-located block.

4. The video encoding method according to claim 3, characterized in that: The deriving, based on the first distance, the second distance and the motion vector of the co-located block, a temporal motion vector candidate of the current block comprises: A product of a ratio between the first distance and the second distance and a motion vector of the co-located block is calculated as a temporal motion vector candidate of the current block.

5. The video encoding method according to claim 1, wherein: The adding the temporal motion vector candidate of the current block to the motion vector candidate list of the current block comprises: Determine whether the temporal motion vector candidate of the current block is repeated with an existing motion vector candidate in the motion vector candidate list of the current block; When the temporal motion vector candidate of the current block does not overlap with the existing motion vector candidate, adding the temporal motion vector candidate of the current block to the motion vector candidate list of the current block; Otherwise, the temporal motion vector candidate of the current block is not added to the motion vector candidate list of the current block.

6. The video encoding method according to claim 1, wherein: The adding the temporal motion vector candidate of the current block to the motion vector candidate list of the current block comprises: Calculate the similarity between the temporal motion vector candidate and the existing motion vector candidates in the motion vector candidate list of the current block; When the similarity is less than a preset threshold, adding the temporal motion vector candidate of the current block to a motion vector candidate list of the current block; Otherwise, the temporal motion vector candidate of the current block is not added to the motion vector candidate list of the current block.

7. A video encoding device, characterized in that: include: A motion vector acquisition module is configured to acquire a motion vector of a co-located block corresponding to the current block in an adjacent frame of the current frame, wherein the adjacent frame is an encoded frame; A temporal motion vector derivation module, configured to derive a temporal motion vector candidate of the current block based on the motion vector of the co-located block; A motion vector adding module, configured to add the temporal motion vector candidate of the current block to the motion vector candidate list of the current block; The encoding module is configured to encode the current block based on the motion vector candidate list of the current block.

8. An electronic device comprising: one or more processors; as well as a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, Wherein, when executing the instructions, the one or more processors are configured to perform the video encoding method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the video encoding method according to any one of claims 1-6.

10. A computer program product having instructions for storing a bitstream, wherein: The bitstream includes: Video data generated by the video encoding method according to any one of claims 1 to 6.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by at least one processor, the video encoding method according to any one of claims 1 to 6 is implemented.

12. A method for generating a bitstream, comprising: A bit stream is generated according to the video encoding method according to any one of claims 1 to 6.

Images