Video encoding and decoding method, bit stream generating and processing method, device, electronic equipment, storage medium and program product

Abstract

The invention provides a video encoding and decoding method, a bit stream generating and processing method and device, electronic equipment, a storage medium and a program product. The video coding method comprises the following steps: dividing a video frame into a plurality of CTUs; determining whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, the CTU-level block copy prediction mode being a mode of copying reconstructed pixels from a co-located CTU in a reference frame; in response to determining that the CTU level block copy prediction mode is enabled for the current CTU, the current CTU is encoded by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU. According to the invention, the size of the code stream can be effectively reduced, and the coding and decoding complexity is reduced while the coding and decoding efficiency is improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of video coding and decoding technology, and more specifically, to a video coding and decoding method, a bit stream generation and processing method, an apparatus, an electronic device, a storage medium, and a program product. Background Art

[0002] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send and receive or otherwise transmit digital video data through a communication network, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, before the video data is transmitted or stored, the video data can be compressed using a video codec according to one or more video codec standards. For example, video codec standards include general video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the inherent redundancy in video data. Video codecs are intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0003] In modern video coding standards, efficient coding of screen content is an important research direction. Screen content usually contains a large number of static areas and repeated patterns, such as text, icons, and graphical interfaces. These characteristics make traditional inter-frame prediction methods ineffective in this scenario. In order to improve coding efficiency, many video coding standards have introduced intra block copy (IBC) technology, which allows a block in the current frame to use other encoded blocks in the same frame as a reference. IBC technology performs well in screen content coding (Screen Content Coding, SCC), which can significantly reduce redundant information and improve compression efficiency. Summary of the invention

[0004] The exemplary embodiments of the present disclosure provide a video encoding and decoding method, a bit stream generation and processing method, an apparatus, an electronic device, a storage medium and a program product, which can effectively reduce the code stream size, improve the encoding and decoding efficiency and reduce the encoding and decoding complexity.

[0005] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, comprising: dividing a video frame into a plurality of CTUs; determining whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, encoding the current CTU by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

[0006] Optionally, the step of determining whether to enable a CTU-level block copy prediction mode for a current CTU in a current frame includes: traversing all reference frames of the current frame to determine whether there is a target reference frame whose source image of a co-located CTU is consistent with the source image of the current CTU; in response to determining that there is the target reference frame, determining to enable the CTU-level block copy prediction mode for the current CTU.

[0007] Optionally, the step of traversing all reference frames of the current frame to determine whether there is a target reference frame whose source image of a co-located CTU is consistent with the source image of the current CTU includes: traversing all reference frames of the current frame to determine whether there is a reference frame whose similarity between the source image of the co-located CTU and the source image of the current CTU exceeds a preset threshold; in response to determining that there is a reference frame whose similarity between the source image of the co-located CTU and the source image of the current CTU exceeds the preset threshold, taking the reference frame whose similarity between the source image of the co-located CTU and the source image of the current CTU is the highest as the target reference frame; in response to determining that there is no reference frame whose similarity between the source image of the co-located CTU and the source image of the current CTU exceeds the preset threshold, determining that the target reference frame does not exist.

[0008] Optionally, it also includes: sending a CTU-level block copy flag at the CTU level, wherein the CTU-level block copy flag indicates whether the CTU-level block copy prediction mode is enabled for the current CTU; in response to the sent CTU-level block copy flag indicating that the CTU-level block copy prediction mode is enabled for the current CTU, sending an index indicating the target reference frame at the CTU level.

[0009] Optionally, the step of sending a CTU-level block copy mark at the CTU level includes: sending a whole-frame block copy mark at the PH level, wherein the whole-frame block copy mark indicates whether the CTU-level block copy prediction mode is enabled for all CTUs in the current frame, and whether the target reference frames of all the CTUs are the same frame; in response to determining that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but the target reference frames of all the CTUs are not the same frame, sending the CTU-level block copy mark at the CTU level; in response to determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frames of all the CTUs are the same frame, sending an index indicating the target reference frame at the PH level.

[0010] Optionally, the step of sending a whole-frame CTU-level block copy mark at the PH level includes: sending a PH-level block copy enable mark at the PH level, wherein the PH-level block copy enable mark indicates whether the CTU-level block copy prediction mode is enabled for the current frame; in response to determining that the CTU-level block copy prediction mode is enabled for the current frame, sending the whole-frame block copy mark at the PH level.

[0011] Optionally, the step of sending a PH level block copy enable mark at the PH level includes: sending an SPS level block copy start mark at the SPS level, wherein the SPS level block copy start mark indicates whether the CTU level block copy prediction mode is turned on for the current sequence; in response to determining that the CTU level block copy prediction mode is turned on for the current sequence, sending the PH level block copy enable mark at the PH level.

[0012] According to a second aspect of an embodiment of the present disclosure, a video decoding method is provided, comprising: receiving an encoded video frame divided into a plurality of CTUs; determining whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, decoding the current CTU by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

[0013] Optionally, it also includes: receiving a CTU-level block copy flag at the CTU level, wherein the CTU-level block copy flag indicates whether the CTU-level block copy prediction mode is enabled for the current CTU; in response to the received CTU-level block copy flag indicating that the CTU-level block copy prediction mode is enabled for the current CTU, receiving an index indicating the target reference frame at the CTU level.

[0014] Optionally, the step of receiving a CTU-level block copy mark at the CTU level includes: receiving a whole-frame block copy mark at the PH level, wherein the whole-frame block copy mark indicates whether the CTU-level block copy prediction mode is enabled for all CTUs in the current frame, and whether the target reference frames of all the CTUs are the same frame; in response to the received whole-frame block copy mark indicating that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but the target reference frames of all the CTUs are not the same frame, receiving the CTU-level block copy mark at the CTU level; in response to the received whole-frame block copy mark indicating that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frames of all the CTUs are the same frame, receiving an index indicating the target reference frame at the PH level.

[0015] Optionally, the step of receiving a whole-frame CTU-level block copy mark at the PH level includes: receiving a PH-level block copy enable mark at the PH level, wherein the PH-level block copy enable mark indicates whether the CTU-level block copy prediction mode is enabled for the current frame; in response to the received PH-level block copy enable mark indicating that the CTU-level block copy prediction mode is enabled for the current frame, receiving the whole-frame block copy mark at the PH level.

[0016] Optionally, the step of receiving a PH level block copy enable mark at the PH level includes: receiving an SPS level block copy start mark at the SPS level, wherein the SPS level block copy start mark indicates whether the CTU level block copy prediction mode is turned on for the current sequence; in response to the received SPS level block copy start mark indicating that the CTU level block copy prediction mode is turned on for the current sequence, receiving the PH level block copy enable mark at the PH level.

[0017] According to a third aspect of an embodiment of the present disclosure, a video encoding device is provided, comprising: a division unit, configured to divide a video frame into multiple CTUs; a determination unit, configured to determine whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; and an encoding unit, configured to, in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, encode the current CTU by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a video decoding device is provided, comprising: a coded video frame receiving unit, configured to receive a coded video frame divided into a plurality of CTUs; a determining unit, configured to determine whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; and a decoding unit, configured to, in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, decode the current CTU by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

[0019] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video encoding method as described above.

[0020] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the video encoding method as described above.

[0021] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the video encoding method as described above when executed by at least one processor.

[0022] According to an eighth aspect of an embodiment of the present disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to the video encoding method as described above.

[0023] According to a ninth aspect of an embodiment of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream includes: video data generated according to the video encoding method as described above.

[0024] According to the tenth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video decoding method as described above.

[0025] According to an eleventh aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the video decoding method as described above.

[0026] According to a twelfth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by at least one processor, implement the video decoding method as described above.

[0027] According to a thirteenth aspect of an embodiment of the present disclosure, a bit stream processing method is provided, comprising: receiving a bit stream; and decoding the bit stream according to the video decoding method as described above.

[0028] According to a fourteenth aspect of an embodiment of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream comprises: video data to be decoded according to the video decoding method as described above.

[0029] According to the video coding and decoding method, bitstream generation and processing method, device, electronic device, storage medium and program product of the exemplary embodiments of the present invention, a new CTU-level prediction mode is proposed. The prediction mode is applied to the CTU-level encoding process. It can implicitly derive operations such as skipping subsequent filtering, effectively reduce the code stream size, and reduce the encoding and decoding complexity while improving the encoding efficiency.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0032] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure;

[0033] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure;

[0034] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure;

[0035] Figure 4 A flowchart showing a video encoding method according to an exemplary embodiment of the present disclosure is shown;

[0036] Figure 5 A flowchart showing a method for determining whether a CTU-level block copy prediction mode is enabled for a current CTU according to an exemplary embodiment of the present disclosure;

[0037] Figure 6A flowchart showing a method for transmitting syntax elements related to a CTU-level block copy prediction mode according to an exemplary embodiment of the present disclosure;

[0038] Figure 7 A flowchart showing a video decoding method according to an exemplary embodiment of the present disclosure is shown;

[0039] Figure 8 A flowchart showing a method for receiving a syntax element related to a CTU-level block copy prediction mode according to an exemplary embodiment of the present disclosure;

[0040] Fig. 9 A structural block diagram of a video encoding device according to an exemplary embodiment of the present disclosure is shown;

[0041] Fig.10 A structural block diagram of a video decoding device according to an exemplary embodiment of the present disclosure is shown;

[0042] Fig.11 A structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown;

[0043] Fig.12 is a diagram illustrating a computing environment coupled with a user interface according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0044] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0045] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0046] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0047] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0048] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0049] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0050] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0051] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0052] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0053] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0054] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0055] Figure 2is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0056] like Figure 2 As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0057] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0058] like Figure 2As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0059] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0060] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0061] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0062] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0063] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0064] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0065] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0066] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0067] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0068] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0069] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0070] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0071] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0072] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0073] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0074] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0075] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0076] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0077] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0078] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0079] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0080] IBC is a coding technology designed for screen content coding, aiming to improve the coding efficiency of screen content. The core idea of ​​IBC technology is similar to traditional inter-frame prediction, but the selection range of its reference block is limited to the current image rather than other reference frames. In IBC, a displacement vector (called a block vector or BV) is used to represent the relative displacement from the position of the current block to the position of the reference block. The reference block is selected from the reconstructed area of ​​the current image. These reference blocks are added to the prediction error to form the reconstructed current block. Specifically, the IBC prediction mode is regarded as a third prediction mode in addition to the intra-frame or inter-frame prediction mode. The encoder finds the best block vector for each CU through block matching technology. The block vector is used to indicate the displacement from the current block to the reference block. Therefore, the IBC prediction mode does not change the overall entropy coding process of the CTU. The IBC prediction mode may also include processes such as residual coding.

[0081] In standards such as HEVC and VVC, each CTU (Coding Tree Unit) requires not only prediction information of the coding block, but also coding filter related parameters, such as Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF). These parameters will increase the video bit rate to a certain extent, and also increase the processing complexity of the encoding and decoding ends. IBC can only refer to the current frame, and it is only a common CU prediction mode that is different from the intra-frame prediction mode and the inter-frame prediction mode, and cannot effectively reduce the codewords related to the filter parameters.

[0082] In order to solve these problems, the present disclosure takes into account that a large number of screen content sequences contain a large number of static scenes, and some non-screen content sequences also have borders and other locations that are static content. Therefore, the present disclosure proposes a CTU-level inter-frame block copying video coding algorithm for static scenes, especially those related to screen content. The core idea of ​​the algorithm is to perform block copying at the CTU level, using the co-located CTU of the reference frame as a reference. Therefore, if a CTU hits the pattern, it is implicitly inferred that: the reconstructed pixels of the CTU are directly copied from the reconstructed pixels of the co-located CTU of the reference frame, without encoding residuals, and implicitly disabling subsequent filtering operations of the CTU, so that each CTU only needs to encode very few bits, which greatly reduces the complexity of the codec and improves compression efficiency. Furthermore, if all CTUs of the current frame hit the pattern, only a few bits need to be marked at the frame level to implement the whole frame copying technology. The following will combine Figures 4 to 12 Provide detailed explanation.

[0083] Figure 4 A flowchart of a video encoding method according to an exemplary embodiment of the present disclosure is shown.

[0084] Reference Figure 4 , in step S401, the video frame is divided into multiple CTUs.

[0085] In step S402 , it is determined whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame.

[0086] The CTU-level block copy prediction mode refers to a mode in which reconstructed pixels are copied from the co-located CTU in the reference frame.

[0087] The following will be combined Figure 5 An exemplary embodiment of step S402 is described below, which will not be expanded here.

[0088] In step S403 , in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, the current CTU is encoded by using reconstructed pixels of a co-located CTU corresponding to the current CTU in the target reference frame as reconstructed pixels of the current CTU.

[0089] Specifically, based on the co-located CTU in the target reference frame, the CTU-level block copy prediction mode is enabled to predict the current CTU in the following way: directly use the reconstructed pixels of the co-located CTU corresponding to the current CTU in the target reference frame as the reconstructed pixels of the current CTU, thereby skipping all subsequent rate-distortion optimization (RDO: Rate-Distortion Optimization) processes (for example, intra-frame prediction, inter-frame prediction, sub-block partitioning, etc.).

[0090] In addition, in response to determining that the CTU-level block copy prediction mode is not enabled for the current CTU, the encoding process of the current CTU is performed according to the existing RDO algorithm.

[0091] In addition, the video encoding method according to the exemplary embodiment of the present disclosure may further include: sending a syntax element related to the CTU level block copy prediction mode, which will be combined with Figure 6 A detailed description is given and will not be expanded here.

[0092] Figure 5 A flowchart of a method for determining whether a CTU-level block copy prediction mode is enabled for a current CTU according to an exemplary embodiment of the present disclosure is shown.

[0093] Reference Figure 5 In step S501, all reference frames of the current frame are traversed to determine whether there is a target reference frame whose source image of the co-located CTU is consistent with the source image of the current CTU.

[0094] As an exemplary embodiment, step S501 may include: traversing all reference frames of the current frame to determine whether there is a reference frame whose source image of the co-located CTU has a similarity with the source image of the current CTU that exceeds a preset threshold; in response to determining that there is a reference frame whose source image of the co-located CTU has a similarity with the source image of the current CTU that exceeds the preset threshold, taking the reference frame with the highest similarity between the source image of the co-located CTU and the source image of the current CTU as the target reference frame; in response to determining that there is no reference frame whose source image of the co-located CTU has a similarity with the source image of the current CTU that exceeds the preset threshold, determining that there is no target reference frame.

[0095] In step S502 , in response to determining that there is a target reference frame, it is determined to enable a CTU-level block copy prediction mode for the current CTU.

[0096] In addition, as an example, the method of determining whether to enable the CTU level block copy prediction mode for the current CTU according to an exemplary embodiment of the present disclosure may further include: in response to determining that there is no target reference frame, determining not to enable the CTU level block copy prediction mode for the current CTU.

[0097] The present disclosure takes into account that: at the encoding end, by analyzing the information source, it is obtained that the co-located CTUs of the reference frame and the current frame are extremely similar, so it can be naturally assumed that the current CTU can obtain the same encoding quality as the co-located CTU of the reference frame without encoding the residual, so it can be implicitly derived that the current CTU does not need to encode the residual and does not need subsequent filtering.

[0098] Figure 6 A flowchart of a method for transmitting syntax elements related to a CTU-level block copy prediction mode according to an exemplary embodiment of the present disclosure is shown.

[0099] Reference Figure 6 In step S601, a CTU-level block copy flag is sent at the CTU level. The CTU-level block copy flag (e.g., ctu_flag) indicates whether the CTU-level block copy prediction mode is enabled for the current CTU. For example, when ctu_flag is 1, it indicates that the CTU-level block copy prediction mode is enabled for the current CTU; when ctu_flag is 0, it indicates that the CTU-level block copy prediction mode is not enabled for the current CTU.

[0100] In step S602, in response to the sent CTU-level block copy flag indicating that the CTU-level block copy prediction mode is enabled for the current CTU, an index indicating a target reference frame is sent at the CTU level.

[0101] As an exemplary embodiment, if the ctu_flag sent for the current CTU is 1, the index of the target reference frame of the current CTU also needs to be sent.

[0102] As an exemplary embodiment, step S601 may specifically include: sending a whole-frame block copy mark at the PH (Picture Header) level; in response to determining that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but the target reference frames of all CTUs are not the same frame, sending a CTU-level block copy mark at the CTU level; in response to determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frames of all CTUs are the same frame, sending an index indicating the target reference frame at the PH level without sending the CTU-level block copy mark and the index of the target reference frame at the CTU level.

[0103] The full-frame block copy flag (e.g., ph_skip_full_flag) indicates whether the CTU-level block copy prediction mode is enabled for all CTUs in the current frame, and whether the target reference frame of all CTUs is the same frame. For example, when ph_skip_full_flag is 1, it indicates that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frame of all CTUs is the same frame; when ph_skip_full_flag is 0, it indicates that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but the target reference frames of all CTUs are not the same frame. If the ph_skip_full_flag sent for the current frame is 1, the index ph_ref_idx of the target reference frame of the current frame needs to be sent as well.

[0104] According to an exemplary embodiment of the present disclosure, if the CTU-level block copy prediction mode is enabled for all CTUs and their target reference frames are the same frame, the final entropy coding process only needs to mark the relevant syntax elements in the image header, thereby directly skipping the coding process of the entire current frame.

[0105] As an exemplary embodiment, the step of sending a whole frame CTU level block copy mark at the PH level may specifically include: sending a PH level block copy enable mark at the PH level; in response to determining that the CTU level block copy prediction mode is enabled for the current frame, sending a whole frame block copy mark at the PH level.

[0106] The PH-level block copy enable flag (e.g., ph_flag) indicates whether the CTU-level block copy prediction mode is enabled for the current frame. For example, when ph_flag is 1, it indicates that the CTU-level block copy prediction mode is enabled for the current frame; when ph_flag is 0, it indicates that the CTU-level block copy prediction mode is not enabled for the current frame.

[0107] As an exemplary embodiment, the step of sending a PH level block copy enable mark at the PH level may specifically include: sending an SPS level block copy enable mark at the SPS (Sequence Parameters Set) level; in response to determining to turn on the CTU level block copy prediction mode for the current sequence, sending a PH level block copy enable mark at the PH level.

[0108] The SPS level block copy on flag (eg, sps_flag) indicates whether the CTU level block copy prediction mode is turned on for the current sequence.

[0109] It should be understood that “enabling CTU-level block copy prediction mode” here means that the CTU-level block copy prediction mode can be enabled (that is, supporting the enabling of the CTU-level block copy prediction mode), but it does not necessarily mean that the CTU-level block copy prediction mode is enabled. Whether the CTU-level block copy prediction mode is enabled is determined by whether each CTU in each frame enables the CTU-level block copy prediction mode.

[0110] The current sequence is the sequence in which the current frame is located. As an exemplary embodiment, whether the CTU-level block copy prediction mode is turned on in the current sequence can be empirically set by the user (e.g., an expert). If the user sets the CTU-level block copy prediction mode to be turned on, sps_flag is 1, otherwise it is 0.

[0111] As an example, a sequence that may contain a large number of static scenes may be set to enable CTU-level block copy prediction mode. For example, a screen content sequence may contain a large number of static scenes, and some non-screen content sequences may also contain static content in positions such as borders.

[0112] To summarize, as an example, at the SPS level, a flag (e.g., sps_flag) is set to mark whether the CTU-level block copy prediction mode is enabled in the current sequence. If the CTU-level block copy prediction mode is enabled in the SPS, it is continued to be marked in the image header of each frame whether the CTU-level block copy prediction mode is enabled for the current frame (e.g., marked using ph_flag). If enabled, it is further marked whether the CTU-level block copy prediction mode is enabled for the entire frame and the target reference frame of all CTUs is the same frame (e.g., marked using ph_skip_full_flag). If so, the index of the target reference frame of the entire frame (ph_ref_idx) is further marked. If ph_flag is 1 and ph_skip_full_flag is 0, when encoding each CTU, mark whether the current CTU enables the CTU-level block copy prediction mode (for example, use ctu_flag for marking). If ctu_flag is 1, continue to mark the index of the target reference frame of the current CTU (ctu_ref_idx) and end the encoding process of the current CTU. If ctu_flag is 0, continue the encoding process in the existing manner.

[0113] It is worth noting that: at the encoding end, when sps_flag is 1, ph_flag will be temporarily marked as 1. When all CTUs of the current frame are encoded, if it is found that any CTU is not similar to the co-located CTU of any reference frame (that is, all CTUs in the current frame do not enable the CTU-level block copy prediction mode), ph_flag will be corrected to 0 to save the codewords for each subsequent CTU encoding. In the present disclosure, as long as sps_flag is 1, ph_flag is allowed to be 1 and 0; as long as ph_flag is 1, ph_skip_full_flag is allowed to be 1 and 0; as long as ph_flag is 1 and ph_skip_full_flag is 0, then ctu_flag is allowed to be 1 and 0. As an example, when sps_flag is 0, ph_flag is implicitly set to 0 (without writing into the bitstream).

[0114] According to an exemplary embodiment of the present disclosure, a certain strategy is adopted at the encoding end to set the value of ctu_flag to achieve optimal coding efficiency. For example, in the above exemplary embodiment, the ctu_flag of each CTU is set at the encoding end through similarity analysis based on the information source.

[0115] The video encoding method according to the exemplary embodiment of the present disclosure can be applied to the luminance component and the chrominance component respectively. In addition, as an exemplary embodiment, the video encoding method can be applied to the luminance component alone or to the chrominance component alone. For example, in the above exemplary embodiment, the luminance and chrominance components are marked with flags separately, rather than being represented by the same flag.

[0116] Figure 7 A flowchart of a video decoding method according to an exemplary embodiment of the present disclosure is shown.

[0117] Reference Figure 7 In step S701, a coded video frame divided into a plurality of CTUs is received.

[0118] In step S702 , it is determined whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame.

[0119] In step S703, in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, the current CTU is decoded by using reconstructed pixels of a co-located CTU corresponding to the current CTU in the target reference frame as reconstructed pixels of the current CTU.

[0120] Specifically, according to the received syntax elements related to the CTU-level block copy prediction mode, it is determined whether the CTU-level block copy prediction mode is enabled for the current CTU in the current frame.

[0121] As an exemplary embodiment, in response to a CTU-level block copy flag received at a CTU level indicating that a CTU-level block copy prediction mode is enabled for a current CTU, it is determined to enable a CTU-level block copy prediction mode for a current CTU.

[0122] As an exemplary embodiment, in response to a whole frame block copy flag received at the PH level indicating that a CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frame of all CTUs is the same frame, it is determined that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame. In this case, the current frame is decoded by using the reconstructed pixels of the target reference frame as the reconstructed pixels of the current frame.

[0123] Figure 8 A flowchart of a method for receiving syntax elements related to a CTU-level block copy prediction mode according to an exemplary embodiment of the present disclosure is shown.

[0124] Reference Figure 8 In step S801, a CTU-level block copy flag is received at the CTU level, wherein the CTU-level block copy flag indicates whether a CTU-level block copy prediction mode is enabled for the current CTU.

[0125] In step S802 , in response to a received CTU-level block copy flag indicating that a CTU-level block copy prediction mode is enabled for a current CTU, an index indicating a target reference frame is received at a CTU level.

[0126] As an exemplary embodiment, step S801 may include: receiving a whole frame block copy mark at the PH level; in response to the received whole frame block copy mark indicating that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but the target reference frames of all CTUs are not the same frame, receiving a CTU-level block copy mark at the CTU level; in response to the received whole frame block copy mark indicating that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frames of all CTUs are the same frame, receiving an index indicating the target reference frame at the PH level. The whole frame block copy mark indicates whether the CTU-level block copy prediction mode is enabled for all CTUs in the current frame, and whether the target reference frames of all CTUs are the same frame.

[0127] As an exemplary embodiment, the step of receiving a whole frame CTU level block copy mark at the PH level includes: receiving a PH level block copy enable mark at the PH level; in response to the received PH level block copy enable mark indicating that the CTU level block copy prediction mode is enabled for the current frame, receiving a whole frame block copy mark at the PH level. The PH level block copy enable mark indicates whether the CTU level block copy prediction mode is enabled for the current frame.

[0128] As an exemplary embodiment, the step of receiving a PH-level block copy enablement flag at the PH level includes: receiving an SPS-level block copy start flag at the SPS level; and receiving a PH-level block copy enablement flag at the PH level in response to the received SPS-level block copy start flag indicating that the CTU-level block copy prediction mode is turned on for the current sequence. The SPS-level block copy start flag indicates whether the CTU-level block copy prediction mode is turned on for the current sequence.

[0129] According to an exemplary embodiment of the present disclosure, a CTU-level inter-frame replication algorithm is proposed, which has no residual coding and implicitly skips the subsequent filtering operation, which can effectively reduce the bit overhead and bit rate of the CTU encoding process, and can also effectively reduce the complexity of encoding and decoding. In addition, when the algorithm is implemented at the encoding end, it can be directly searched through the source without relying on the reconstructed image. Therefore, the encoding end is very concise, and can make full use of the encoding end resources and maximize the encoding parallelism.

[0130] Fig. 9 A structural block diagram of a video encoding apparatus according to an exemplary embodiment of the present disclosure is shown.

[0131] like Fig. 9 As shown, the video encoding device 900 according to an exemplary embodiment of the present disclosure includes: a dividing unit 901, a determining unit 902, and an encoding unit 903.

[0132] Specifically, the dividing unit 901 is configured to divide the video frame into a plurality of CTUs.

[0133] The determining unit 902 is configured to determine whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame.

[0134] The encoding unit 903 is configured to, in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, encode the current CTU by using the reconstructed pixels of the co-located CTU corresponding to the current CTU in the target reference frame as the reconstructed pixels of the current CTU.

[0135] As an exemplary embodiment, the determination unit 902 may be configured to: traverse all reference frames of the current frame to determine whether there is a target reference frame whose source image of the co-located CTU is consistent with the source image of the current CTU; in response to determining that the target reference frame exists, determine to enable the CTU-level block copy prediction mode for the current CTU.

[0136] As an exemplary embodiment, the determination unit 902 may be configured to: traverse all reference frames of the current frame to determine whether there is a reference frame whose source image of the co-located CTU has a similarity with the source image of the current CTU that exceeds a preset threshold; in response to determining that there is a reference frame whose source image of the co-located CTU has a similarity with the source image of the current CTU that exceeds the preset threshold, use the reference frame whose source image of the co-located CTU has the highest similarity with the source image of the current CTU as the target reference frame; in response to determining that there is no reference frame whose source image of the co-located CTU has a similarity with the source image of the current CTU that exceeds the preset threshold, determine that the target reference frame does not exist.

[0137] As an exemplary embodiment, the encoding unit 903 may be configured to: send a CTU-level block copy flag at the CTU level, wherein the CTU-level block copy flag indicates whether the CTU-level block copy prediction mode is enabled for the current CTU; in response to the sent CTU-level block copy flag indicating that the CTU-level block copy prediction mode is enabled for the current CTU, send an index indicating the target reference frame at the CTU level.

[0138] As an exemplary embodiment, the encoding unit 903 may be configured to: send a whole-frame block copy mark at the PH level, wherein the whole-frame block copy mark indicates whether the CTU-level block copy prediction mode is enabled for all CTUs in the current frame, and whether the target reference frames of all the CTUs are the same frame; in response to determining that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but the target reference frames of all the CTUs are not the same frame, send the CTU-level block copy mark at the CTU level; in response to determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frames of all the CTUs are the same frame, send an index indicating the target reference frame at the PH level.

[0139] As an exemplary embodiment, the encoding unit 903 may be configured to: send a PH-level block copy enable mark at the PH level, wherein the PH-level block copy enable mark indicates whether the CTU-level block copy prediction mode is enabled for the current frame; in response to determining that the CTU-level block copy prediction mode is enabled for the current frame, send the whole frame block copy mark at the PH level.

[0140] As an exemplary embodiment, the encoding unit 903 may be configured to: send an SPS-level block copy enable mark at the SPS level, wherein the SPS-level block copy enable mark indicates whether the CTU-level block copy prediction mode is turned on for the current sequence; in response to determining that the CTU-level block copy prediction mode is turned on for the current sequence, send a PH-level block copy enable mark at the PH level.

[0141] Fig.10 A structural block diagram of a video decoding device according to an exemplary embodiment of the present disclosure is shown.

[0142] like Fig.10 As shown, the video decoding device 1000 according to an exemplary embodiment of the present disclosure includes: a coded video frame receiving unit 1001 , a determining unit 1002 , and a decoding unit 1003 .

[0143] Specifically, the coded video frame receiving unit 1001 is configured to receive a coded video frame divided into a plurality of CTUs.

[0144] The determining unit 1002 is configured to determine whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame.

[0145] The decoding unit 1003 is configured to, in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, decode the current CTU by using the reconstructed pixels of the co-located CTU corresponding to the current CTU in the target reference frame as the reconstructed pixels of the current CTU.

[0146] As an exemplary embodiment, the decoding unit 1003 may be configured to: receive a CTU-level block copy flag at a CTU level, wherein the CTU-level block copy flag indicates whether the CTU-level block copy prediction mode is enabled for the current CTU; and receive an index indicating the target reference frame at the CTU level in response to the received CTU-level block copy flag indicating that the CTU-level block copy prediction mode is enabled for the current CTU.

[0147] As an exemplary embodiment, the decoding unit 1003 may be configured to: receive a whole frame block copy mark at the PH level, wherein the whole frame block copy mark indicates whether the CTU-level block copy prediction mode is enabled for all CTUs in the current frame, and whether the target reference frames of all the CTUs are the same frame; in response to the received whole frame block copy mark indicating that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but the target reference frames of all the CTUs are not the same frame, receive the CTU-level block copy mark at the CTU level; in response to the received whole frame block copy mark indicating that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frames of all the CTUs are the same frame, receive an index indicating the target reference frame at the PH level.

[0148] As an exemplary embodiment, the decoding unit 1003 may be configured to: receive a PH-level block copy enable mark at the PH level, wherein the PH-level block copy enable mark indicates whether the CTU-level block copy prediction mode is enabled for the current frame; and receive the whole-frame block copy mark at the PH level in response to the received PH-level block copy enable mark indicating that the CTU-level block copy prediction mode is enabled for the current frame.

[0149] As an exemplary embodiment, the decoding unit 1003 may be configured to: receive an SPS level block copy enable mark at the SPS level, wherein the SPS level block copy enable mark indicates whether the CTU level block copy prediction mode is enabled for the current sequence; and receive a PH level block copy enable mark at the PH level in response to the received SPS level block copy enable mark indicating that the CTU level block copy prediction mode is enabled for the current sequence.

[0150] Regarding the video encoding device 900 and the video decoding device 1000 in the above embodiments, the specific manner in which each unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0151] In addition, it should be understood that the various units in the video encoding device 900 and the video decoding device 1000 according to the exemplary embodiments of the present disclosure may be implemented as hardware components and / or software components. Those skilled in the art may implement the various units, for example, using a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), according to the processing performed by the defined various units.

[0152] Fig.11 FIG. 1 is a block diagram showing a structure of an electronic device according to an exemplary embodiment of the present disclosure. Fig.11 The electronic device 1100 includes: at least one memory 1101 and at least one processor 1102, wherein the at least one memory 1101 stores a set of computer executable instructions, and when the computer executable instruction set is executed by the at least one processor 1102, the video encoding method and / or the video decoding method described in the above exemplary embodiments is executed.

[0153] As an example, the electronic device 1100 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device 1100 is not necessarily a single electronic device, but may also be any device or circuit collection capable of executing the above-mentioned instructions (or instruction sets) individually or in combination. The electronic device 1100 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0154] In the electronic device 1100, the processor 1102 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor 1102 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0155] The processor 1102 may execute instructions or codes stored in the memory 1101, wherein the memory 601 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0156] The memory 1101 may be integrated with the processor 1102, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 1101 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 1101 and the processor 1102 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 1102 can read files stored in the memory.

[0157] In addition, the electronic device 1100 may also include a multimedia resource display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 1100 may be connected to each other via a bus and / or a network.

[0158] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one processor, the at least one processor is prompted to perform the video encoding method and / or the video decoding method as described in the above exemplary embodiments. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0159] Fig.12 A computing environment 1210 is shown coupled to a user interface 1250. The computing environment 1210 may be part of a data processing server. The computing environment 1210 includes a processor 1220, a memory 1230, and an input / output (I / O) interface 1240.

[0160] The processor 1220 generally controls the overall operation of the computing environment 1210, such as operations associated with display, data acquisition, data communication, and image processing. The processor 1220 may include one or more processors for executing instructions to perform all or some of the steps in the above method. In addition, the processor 1220 may include one or more modules that facilitate the interaction between the processor 1220 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0161] The memory 1230 is configured to store various types of data to support the operation of the computing environment 1210. The memory 1230 may include predetermined software 1232. Examples of such data include instructions for any application or method operating on the computing environment 1210, video data sets, image data, etc. The memory 1230 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0162] I / O interface 1240 provides an interface between processor 1220 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 1240 may be coupled to an encoder and a decoder.

[0163] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 1230 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 1220 in a computing environment 1210 to perform the above method. In one example, the plurality of programs can be executed by a processor 1220 in a computing environment 1210 to (for example, from Figure 2The video encoder 20 in the computing environment 1210 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1220 in the computing environment 1210 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 1220 in the computing environment 1210 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 1220 in the computing environment 1210 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0164] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0165] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 1220); and a non-transitory computer-readable storage medium or memory 1230 in which multiple programs that can be executed by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0166] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes the encoded video information generated by the above encoding method or the encoded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 1230 is also provided, wherein the plurality of programs can be executed by a processor 1220 in a computing environment 1210 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0167] In an embodiment, the computing environment 1210 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0168] In an embodiment, a method for storing a bit stream is also provided, comprising: storing the bit stream on a digital storage medium, wherein the bit stream comprises encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0169] In an embodiment, a method for transmitting a bit stream generated by the above encoder is also provided. In an embodiment, a method for receiving a bit stream to be decoded by the above decoder is also provided.

[0170] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0171] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0172] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video encoding method, characterized in that: include: Divide the video frame into multiple CTUs; Determine whether to enable a CTU-level block copy prediction mode for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; In response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, the current CTU is encoded by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

2. The video encoding method according to claim 1, characterized in that The step of determining whether to enable the CTU-level block copy prediction mode for the current CTU in the current frame comprises: Traversing all reference frames of the current frame, and determining whether there is a target reference frame whose source image of a co-located CTU is consistent with the source image of the current CTU; In response to determining that the target reference frame exists, determining to enable the CTU-level block copy prediction mode for the current CTU.

3. The video encoding method according to claim 2, characterized in that: The step of traversing all reference frames of the current frame and determining whether there is a target reference frame whose source image of a co-located CTU is consistent with the source image of the current CTU comprises: Traversing all reference frames of the current frame, and determining whether there is a reference frame whose source image of a co-located CTU has a similarity with the source image of the current CTU exceeding a preset threshold; In response to determining that there is a reference frame whose similarity between the source image of the co-located CTU and the source image of the current CTU exceeds the preset threshold, taking the reference frame whose similarity between the source image of the co-located CTU and the source image of the current CTU is the highest as the target reference frame; In response to determining that there is no reference frame whose similarity between a source image of a co-located CTU and a source image of the current CTU exceeds the preset threshold, it is determined that there is no target reference frame.

4. The video encoding method according to claim 1, characterized in that: Also includes: Sending a CTU level block copy flag at a CTU level, wherein the CTU level block copy flag indicates whether the CTU level block copy prediction mode is enabled for the current CTU; In response to the CTU-level block copy flag being sent indicating that the CTU-level block copy prediction mode is enabled for the current CTU, an index indicating the target reference frame is sent at the CTU level.

5. The video encoding method according to claim 4, characterized in that: The step of sending a CTU-level block copy mark at the CTU level comprises: Sending a whole frame block copy flag at the PH level, wherein the whole frame block copy flag indicates whether the CTU level block copy prediction mode is enabled for all CTUs in the current frame, and whether target reference frames of all CTUs are the same frame; In response to determining that the CTU-level block copy prediction mode is not enabled for all CTUs in the current frame, or determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame but target reference frames of all CTUs are not the same frame, sending the CTU-level block copy flag at the CTU level; In response to determining that the CTU-level block copy prediction mode is enabled for all CTUs in the current frame and the target reference frames of all the CTUs are the same frame, an index indicating the target reference frame is sent at the PH level.

6. The video encoding method according to claim 5, characterized in that: The step of sending a whole frame CTU level block copy mark at the PH level includes: sending a PH-level block copy enable flag at the PH level, wherein the PH-level block copy enable flag indicates whether the CTU-level block copy prediction mode is enabled for the current frame; In response to determining that the CTU level block copy prediction mode is enabled for the current frame, the whole frame block copy flag is sent at the PH level.

7. The video encoding method according to claim 6, characterized in that: The step of sending a PH-level block copy enablement mark at the PH level includes: Sending an SPS level block copy enable flag at the SPS level, wherein the SPS level block copy enable flag indicates whether to enable the CTU level block copy prediction mode for the current sequence; In response to determining that the CTU level block copy prediction mode is turned on for the current sequence, a PH level block copy enable flag is sent at the PH level.

8. A video decoding method, characterized in that: include: Receiving an encoded video frame divided into a plurality of CTUs; Determine whether to enable a CTU-level block copy prediction mode for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; In response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, the current CTU is decoded by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

9. A video encoding device, characterized in that: include: A division unit configured to divide a video frame into a plurality of CTUs; A determining unit configured to determine whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; The encoding unit is configured to, in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, encode the current CTU by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

10. A video decoding device, characterized in that: include: A coded video frame receiving unit configured to receive a coded video frame divided into a plurality of CTUs; A determining unit configured to determine whether a CTU-level block copy prediction mode is enabled for a current CTU in a current frame, wherein the CTU-level block copy prediction mode refers to a mode of copying reconstructed pixels from a co-located CTU in a reference frame; A decoding unit is configured to, in response to determining that the CTU-level block copy prediction mode is enabled for the current CTU, decode the current CTU by using reconstructed pixels of a co-located CTU corresponding to the current CTU in a target reference frame as reconstructed pixels of the current CTU.

11. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video encoding method according to any one of claims 1 to 7 and / or the video decoding method according to claim 8.

12. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video encoding method according to any one of claims 1 to 7 and / or the video decoding method according to claim 8.

13. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by at least one processor, the video encoding method according to any one of claims 1 to 7 and / or the video decoding method according to claim 8 are implemented.

14. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video encoding method according to any one of claims 1 to 7.

15. A bit stream processing method, characterized in that: include: Receive a bit stream; The bit stream is decoded according to the video decoding method of claim 8.

Images