Video coding method and device

Abstract

The invention relates to a video coding method and device. The video coding method comprises the following steps: determining a luminance intra-frame reference cost, a luminance inter-frame reference cost, a chroma intra-frame reference cost and a chroma inter-frame reference cost of a current coding unit; determining a propagation cost of the current coding unit based on the luminance intra-frame reference cost, the luminance inter-frame reference cost, the chroma intra-frame reference cost and the chroma inter-frame reference cost; determining a quantization parameter offset of the current coding unit based on the luminance intra-frame reference cost, the chroma intra-frame reference cost and the propagation cost; and performing quantization processing on the current coding unit based on the quantization parameter offset, and encoding the current coding unit based on the quantization processing. According to the video coding method and device disclosed by the invention, the quantization parameter offset of the current coding unit can be determined through the propagation cost based on the luminance intra-frame reference cost, the chroma intra-frame reference cost, the chroma intra-frame reference cost and the chroma inter-frame reference cost, so that the coding performance is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video technology, and more specifically, to a video encoding method and device. Background Art

[0002] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send and receive or otherwise transmit digital video data through a communication network, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, before the video data is transmitted or stored, the video data can be compressed using a video codec according to one or more video codec standards. For example, video codec standards include general video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the inherent redundancy in video data. Video codecs are intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0003] In the related art, rate control plays an important role in actual video coding systems. The purpose of rate control is to minimize distortion under rate constraints. However, the loss propagation model in the pre-analysis process of open source encoders (e.g., x264 encoder and x265 encoder) is inaccurate, resulting in limited encoding compression performance. Summary of the invention

[0004] An exemplary embodiment of the present disclosure is to provide a video encoding method and apparatus to at least solve the problems of video encoding in the related art, but may not solve any of the above problems.

[0005] According to an exemplary embodiment of the present disclosure, a video encoding method is provided, including: determining a luma intra-frame reference cost and a luma inter-frame reference cost, a chroma intra-frame reference cost and a chroma inter-frame reference cost of a current coding unit; determining a propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost and the chroma intra-frame reference cost and the chroma inter-frame reference cost; determining a quantization parameter offset of the current coding unit based on the luma intra-frame reference cost, the chroma intra-frame reference cost and the propagation cost; quantizing the current coding unit based on the quantization parameter offset, and encoding the current coding unit based on the quantization.

[0006] Optionally, determining the propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost as well as the chroma intra-frame reference cost and the chroma inter-frame reference cost may include: determining the luma propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost; determining the chroma propagation cost of the current coding unit based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; determining the propagation cost based on the luma propagation cost and the chroma propagation cost.

[0007] Optionally, determining the propagation cost based on the luminance propagation cost and the chrominance propagation cost may include: determining the propagation cost based on a weighted calculation result of the luminance propagation cost and the chrominance propagation cost.

[0008] Optionally, determining the chroma propagation cost of the current coding unit based on the chroma intra-frame reference cost and the chroma inter-frame reference cost may include: determining the chroma reference ratio transferred from the current coding unit to the reference frame based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; determining the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit and the overlapping area of ​​the current coding unit and the reference frame.

[0009] Optionally, determining the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area of ​​the current coding unit and the reference frame may include: determining the amount of transmitted chroma information from the current coding unit to the reference frame based on the chroma intra-frame reference cost, the chroma initial propagation cost of the current coding unit, and the chroma reference ratio; determining the chroma propagation cost based on the chroma initial propagation cost, the transmitted information amount and the overlapping area.

[0010] Optionally, the determining of the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit and the overlapping area may include: based on the current coding unit being referenced multiple times, performing the following steps to determine the chroma propagation cost: for the first reference among the multiple references, determining a first amount of transferred information transmitted from the current coding unit to the reference frame at the first reference among the multiple references based on the chroma intra-frame reference cost, the chroma initial propagation cost and the chroma reference ratio, and determining a first chroma propagation cost of the current coding unit based on the chroma initial propagation cost, the first amount of transferred information and the overlapping area; for the second reference among the multiple references Consider, based on the chroma frame reference cost, the first chroma propagation cost and the chroma reference ratio, determine the second amount of transferred information transmitted from the current coding unit to the reference frame when the second reference is made in the multiple references, and determine the second chroma propagation cost of the current coding unit based on the first chroma propagation cost, the second amount of transferred information and the area of ​​the overlapping region; for subsequent references in the multiple references, iteratively replace the chroma propagation cost and the amount of transferred information with the chroma propagation cost and the amount of transferred information determined for the previous reference and perform the above steps until the last chroma propagation cost of the current coding unit is determined for the last reference in the multiple references, and determine the chroma propagation cost based on the last chroma propagation cost.

[0011] Optionally, determining the quantization parameter offset of the current coding unit based on the luma intra-frame reference cost, the chroma intra-frame reference cost and the propagation cost may include: determining the intra-frame reference cost of the current coding unit based on the luma intra-frame reference cost and the chroma intra-frame reference cost; determining the quantization parameter offset based on the intra-frame reference cost and the propagation cost.

[0012] Optionally, encoding the current coding unit based on the quantization process may include: determining a chroma quantization parameter offset and a luminance quantization parameter offset of the current coding unit; determining a chroma encoding rate of the current coding unit based on the chroma quantization parameter offset and the luminance quantization parameter offset; encoding the current coding unit based on the quantization process and the chroma encoding rate.

[0013] Optionally, determining the chroma encoding rate of the current coding unit based on the chroma quantization parameter offset and the luminance quantization parameter offset may include: determining a rate-distortion cost function related to the chroma encoding rate based on the chroma quantization parameter offset and the luminance quantization parameter offset; and determining the chroma encoding rate based on the rate-distortion cost function.

[0014] According to an exemplary embodiment of the present disclosure, a video encoding device is provided, including: a reference cost determination unit, configured to determine a luma intra-frame reference cost and a luma inter-frame reference cost, a chroma intra-frame reference cost and a chroma inter-frame reference cost of a current coding unit; a propagation cost determination unit, configured to determine the propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost, and the chroma intra-frame reference cost and the chroma inter-frame reference cost; an offset determination unit, configured to determine a quantization parameter offset of the current coding unit based on the luma intra-frame reference cost, the chroma intra-frame reference cost and the propagation cost; and an encoding unit, configured to quantize the current coding unit based on the quantization parameter offset, and encode the current coding unit based on the quantization process.

[0015] Optionally, the propagation cost determination unit can be configured to: determine the luminance propagation cost of the current coding unit based on the luminance intra-frame reference cost and the luminance inter-frame reference cost; determine the chrominance propagation cost of the current coding unit based on the chrominance intra-frame reference cost and the chrominance inter-frame reference cost; determine the propagation cost based on the luminance propagation cost and the chrominance propagation cost.

[0016] Optionally, the propagation cost determining unit may be configured to determine the propagation cost based on a weighted calculation result of the luminance propagation cost and the chrominance propagation cost.

[0017] Optionally, the propagation cost determination unit may be configured to: determine the chroma reference ratio passed from the current coding unit to the reference frame based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; determine the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area of ​​the current coding unit and the reference frame.

[0018] Optionally, the propagation cost determination unit can be configured to: determine the amount of transmitted chrominance information from the current coding unit to the reference frame based on the chrominance intra-frame reference cost, the chrominance initial propagation cost of the current coding unit and the chrominance reference ratio; determine the chrominance propagation cost based on the chrominance initial propagation cost, the amount of transmitted information and the overlapping area.

[0019] Optionally, the propagation cost determination unit may be configured to: based on the current coding unit being referenced multiple times, perform the following steps to determine the chroma propagation cost: for the first reference among the multiple references, determine the first amount of transmitted information transmitted from the current coding unit to the reference frame at the first reference among the multiple references based on the chroma intra-frame reference cost, the chroma initial propagation cost and the chroma reference ratio, and determine the first chroma propagation cost of the current coding unit based on the chroma initial propagation cost, the first amount of transmitted information and the overlapping area; for the second reference among the multiple references, determine the first chroma propagation cost based on the chroma intra-frame reference cost, the first chroma propagation cost and the chrominance reference ratio, determine the second amount of information transferred from the current coding unit to the reference frame when the second reference is made in the multiple references, and determine the second chrominance propagation cost of the current coding unit based on the first chrominance propagation cost, the second amount of information transferred, and the area of ​​the overlapping region; for subsequent references in the multiple references, iteratively replace the chrominance propagation cost and the amount of information transferred with the chrominance propagation cost and the amount of information transferred determined for the previous reference and perform the above steps until the last chrominance propagation cost of the current coding unit is determined for the last reference in the multiple references, and determine the chrominance propagation cost based on the last chrominance propagation cost.

[0020] Optionally, the offset determination unit may be configured to: determine the intra-frame reference cost of the current coding unit based on the luma intra-frame reference cost and the chroma intra-frame reference cost; and determine the quantization parameter offset based on the intra-frame reference cost and the propagation cost.

[0021] Optionally, the encoding unit may be configured to: determine a chroma quantization parameter offset and a luminance quantization parameter offset of the current encoding unit; determine a chroma encoding rate of the current encoding unit based on the chroma quantization parameter offset and the luminance quantization parameter offset; and encode the current encoding unit based on the quantization process and the chroma encoding rate.

[0022] Optionally, the encoding unit may be configured to: determine a rate-distortion cost function related to the chroma encoding rate based on the chroma quantization parameter offset and the luminance quantization parameter offset; and determine the chroma encoding rate based on the rate-distortion cost function.

[0023] According to an exemplary embodiment of the present disclosure, an electronic device is provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a video encoding method according to an exemplary embodiment of the present disclosure.

[0024] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor of an electronic device, the electronic device performs a video encoding method according to an exemplary embodiment of the present disclosure.

[0025] According to an exemplary embodiment of the present disclosure, a computer program product is provided, including a computer program / instruction. When the computer program / instruction is executed by a processor, the video encoding method according to the exemplary embodiment of the present disclosure is implemented.

[0026] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0027] By determining the quantization parameter offset of the current coding unit based on the luma frame intra-reference cost, the chroma frame intra-reference cost, and the propagation cost based on the chroma frame intra-reference cost and the chroma frame inter-reference cost, the accuracy of the quantization parameter offset of the current coding unit can be improved, and the current coding unit is quantized based on the highly accurate quantization parameter offset, and the current coding unit is encoded based on the quantization process, thereby improving the encoding performance.

[0028] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0030] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0031] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0032] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0033] Figure 4 A flowchart of a video encoding method according to an exemplary embodiment of the present disclosure is shown.

[0034] Figure 5 A block diagram of a video encoding apparatus according to an exemplary embodiment of the present disclosure is shown.

[0035] Figure 6is a diagram illustrating a computing environment coupled with a user interface according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0036] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the drawings are used to distinguish objects, and are not used to describe any specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0038] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0039] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0040] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0041] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0042] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0043] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0044] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0045] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0046] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0047] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0048] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0049] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0050] like Figure 2 As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0051] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0052] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0053] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0054] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0055] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0056] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0057] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0058] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0059] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0060] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0061] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0062] In some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0063] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0064] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0065] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0066] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0067] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0068] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0069] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0070] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0071] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0072] The video coding standard divides image frames into three types according to different functions and compression efficiency: intra-frame (I-frame), predicted-frame (P-frame) and bidirectional-frame (B-frame). Except for I-frame, which is self-reconstructed, other types of image frames refer to other image frames for predictive coding to improve the coding efficiency of the encoder.

[0073] At present, video coding standards are all based on blocks (e.g., coding units), and each image frame in the video is usually divided into blocks of different sizes, and then each block is encoded separately. Since P frames and B frames need to use already encoded image frames as reference image frames, that is, search for corresponding encoded blocks in the reference image frames to perform inter-frame prediction on the encoded blocks in the current image frame to reduce information redundancy between adjacent image frames. For inter-frame prediction, the quality of the reference image frame has a direct impact on the coding efficiency of the current image frame.

[0074] However, whether it is for standard reference software such as HEVC and VVC or encoders such as x264 and x265, the frame-level quantization parameter (QP) offset is determined by configuration or parameters before encoding begins, that is, it is set to a constant value, such as -1, and there is no adaptive adjustment, which makes the algorithm in the encoder unable to fully exert its advantages.

[0075] The coding unit tree is an objective coding quality optimization tool in video coding. The basic principle of this tool is to adjust the quantization parameter according to the degree of reference of the current coding unit during preprocessing, that is, in a multi-layer reference frame structure, a smaller quantization parameter is used for a coding unit with a high degree of reference, and vice versa, a larger quantization parameter is used for a coding unit with a low degree of reference, so as to achieve the purpose of improving the objective quality of coding. Specifically, the coding unit tree usually adjusts the quantization parameter by reducing the original quantization parameter, but the reduction in the quantization parameter is larger for a coding unit with a high degree of reference, and smaller for a coding unit with a low degree of reference.

[0076] However, the loss propagation models of coding quality optimization tools such as coding unit trees in the pre-analysis process are only based on brightness information for pre-analysis, which results in the inaccuracy of the loss propagation model and limits the coding compression performance.

[0077] Therefore, the present disclosure provides a video encoding method, which can improve the accuracy of the quantization parameter offset of the current coding unit by determining the quantization parameter offset of the current coding unit based on the luma frame intra-reference cost, the chroma frame intra-reference cost, and the propagation cost based on the chroma frame intra-reference cost and the chroma frame inter-reference cost, and quantize the current coding unit based on the highly accurate quantization parameter offset, and encode the current coding unit based on the quantization process, thereby improving the encoding performance.

[0078] Next, we will refer to Figure 4 and Figure 5 A video encoding method and apparatus according to exemplary embodiments of the present disclosure are described in detail.

[0079] Figure 4 A flowchart of a video encoding method according to an exemplary embodiment of the present disclosure is shown.

[0080] Reference Figure 4, in step S401, the luma intra-frame reference cost and the luma inter-frame reference cost, the chroma intra-frame reference cost and the chroma inter-frame reference cost of the current coding unit are determined. Here, the luma intra-frame reference cost may be greater than or equal to the luma inter-frame reference cost, and the chroma intra-frame reference cost may be greater than or equal to the chroma inter-frame reference cost. Here, the luma intra-frame reference cost and the luma inter-frame reference cost, the chroma intra-frame reference cost and the chroma inter-frame reference cost of the current coding unit may be obtained in the encoding pre-analysis stage before video encoding. In the encoding pre-analysis stage, before obtaining the luma intra-frame reference cost and the luma inter-frame reference cost, the chroma intra-frame reference cost and the chroma inter-frame reference cost of the current coding unit, the image frame where the current coding unit is located or the entire current image may be downsampled, for example, but not limited to, downsampling by half. For example, a 256×256 image is downsampled to a 128×128 image.

[0081] In an exemplary embodiment of the present disclosure, a chroma intra-frame reference cost refers to a chroma component reference cost of a current coding unit referring to the current coding unit itself in a pre-encoding analysis stage, a chroma inter-frame reference cost refers to a chroma component reference cost of a coding unit in other image frames other than the image frame where the current coding unit is located in a pre-encoding analysis stage, a luma intra-frame reference cost refers to a luma component reference cost of a current coding unit referring to the current coding unit itself in a pre-encoding analysis stage, and a luma inter-frame reference cost refers to a luma component reference cost of a coding unit in other image frames other than the image frame where the current coding unit is located in a pre-encoding analysis stage. For example, in a pre-encoding analysis stage, a reference cost may be determined by comparing a residual image between a reference unit and a prediction unit of a current coding unit, and applying an absolute transform error and algorithm to the residual image.

[0082] As an example, if the current coding unit is called coding unit A, and coding unit A is predicted with reference to coding unit A to obtain prediction unit A1, then when calculating the chroma intra-frame reference cost of coding unit A with reference to coding unit A, the chroma residual image 1 between coding unit A and prediction unit A1 can be calculated first, and then the chroma intra-frame reference cost is determined by applying the absolute transform error and algorithm to the chroma residual image 1; when calculating the luminance intra-frame reference cost of coding unit A with reference to coding unit A, the luminance residual image 1 between coding unit A and prediction unit A1 can be calculated first, and then the luminance intra-frame reference cost is determined by applying the absolute transform error and algorithm to the luminance residual image 1.

[0083] As an example, if the current coding unit is called coding unit A, and coding unit A obtains prediction unit A2 by referring to coding unit C outside the image frame where coding unit A is located, then when calculating the chroma inter-frame reference cost of coding unit A referring to coding unit C, the chroma residual image 2 between coding unit C and prediction unit A2 can be calculated first, and then the chroma inter-frame reference cost can be determined by applying the absolute transformation error and algorithm to the chroma residual image 2; when calculating the luminance inter-frame reference cost of coding unit A referring to coding unit C, the luminance residual image 2 between coding unit C and prediction unit A2 can be calculated first, and then the luminance inter-frame reference cost can be determined by applying the absolute transformation error and algorithm to the luminance residual image 2.

[0084] In step S402, a propagation cost of the current coding unit is determined based on the luma intra-frame reference cost and the luma inter-frame reference cost and the chroma intra-frame reference cost and the chroma inter-frame reference cost. Here, the propagation cost represents information propagated to other coding units when the current coding unit is used as a reference coding unit by other coding units in the encoding stage.

[0085] To know the contribution of the current coding unit to other coding units (or frames) that will be encoded with reference to the current coding unit in the future, it is necessary to infer how much information comes from the current coding unit from other coding units (or frames) that will be encoded with reference to the current coding unit in the future. Since other coding units (or frames) that will be encoded with reference to the current coding unit in the future have not been encoded yet, it is necessary to predict through, for example, forward prediction (lookahead). Forward prediction estimates the reference cost for a certain number of uncoded units (or frames) and determines the propagation cost based on the estimated reference cost.

[0086] In an exemplary embodiment of the present disclosure, determining the propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost as well as the chroma intra-frame reference cost and the chroma inter-frame reference cost may include: determining the luma propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost; determining the chroma propagation cost of the current coding unit based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; determining the propagation cost based on the luma propagation cost and the chroma propagation cost, thereby improving the accuracy of the propagation cost by introducing the chroma propagation cost into the propagation cost.

[0087] In an exemplary embodiment of the present disclosure, determining the propagation cost based on the luminance propagation cost and the chrominance propagation cost may include: determining the propagation cost based on a weighted calculation result of the luminance propagation cost and the chrominance propagation cost, thereby improving the accuracy of the propagation cost.

[0088] As an example, the formula To determine the propagation cost of the current coding unit. Here, C propagate represents the transmission cost, represents the brightness propagation cost, represents the chrominance propagation cost, and α and β represent predetermined parameters. For example, α may represent the weighting coefficient of the luminance propagation cost, and β may represent the weighting coefficient of the chrominance propagation cost.

[0089] In an exemplary embodiment of the present disclosure, determining the chroma propagation cost of the current coding unit based on the chroma intra-frame reference cost and the chroma inter-frame reference cost may include: determining a chroma reference ratio transferred from the current coding unit to a reference frame based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; determining the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area of ​​the current coding unit and the reference frame, thereby obtaining the chroma propagation cost. Here, the chroma reference ratio represents the proportion of chroma information transferred from the current coding unit to the reference frame. A larger chroma reference ratio indicates a larger proportion of chroma information transferred from the current coding unit to the reference frame, and a smaller chroma reference ratio indicates a smaller proportion of chroma information transferred from the current coding unit to the reference frame.

[0090] As an example, the formula To determine the chrominance reference ratio passed from the current coding unit to the reference frame here, represents the chroma intra reference cost, represents the chroma inter-frame reference cost. If Much smaller than This means that the images before and after the video are very similar, and the reference cost of using the inter-frame mode is very small. If the current coding unit is encoded with high quality, the reference cost of subsequent coding units will be very small. In this case, the chrominance reference ratio is relatively large, resulting in a relatively large propagation cost, that is, the importance of the current coding unit is very high. and The difference is small, indicating that the similarity between the images before and after the video is very low, and the reference cost of using the inter-frame mode is very high. In this case, the chromaticity reference ratio is relatively small, resulting in a relatively small propagation cost, that is, the importance of the current coding unit is very low. For example, if yes 80% of is 0.2, indicating that inter-frame prediction saves 20% of the coding cost, and 20% of the chrominance information of the current coding unit refers to the reference frame. yes 20%, then It is 0.8, which means that inter-frame prediction saves 80% of the coding cost, and 80% of the chrominance information of the current coding unit refers to the reference frame.

[0091] In an exemplary embodiment of the present disclosure, determining the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area between the current coding unit and the reference frame may include: determining the amount of chroma information transferred from the current coding unit to the reference frame based on the chroma intra-frame reference cost, the chroma initial propagation cost of the current coding unit, and the chroma reference ratio; determining the chroma propagation cost based on the chroma initial propagation cost, the amount of information transferred, and the overlapping area, thereby improving the accuracy of the chroma propagation cost. Here, the amount of chroma information transferred represents the amount of chroma information transferred from the current coding unit to the reference frame. The larger the chroma reference ratio, the larger the amount of chroma information transferred, and the more chroma information is transferred from the current coding unit to the reference frame. The smaller the chroma reference ratio, the smaller the amount of chroma information transferred, and the less chroma information is transferred from the current coding unit to the reference frame.

[0092] As an example, the formula To determine the amount of chrominance information transferred from the current coding unit to the reference frame here, represents the chroma intra reference cost, represents the chromaticity propagation cost, Indicates the colorimetric reference ratio. For example, if is 0.2, then for have The chrominance information is transferred from the current coding unit to the reference frame.

[0093] In an exemplary embodiment of the present disclosure, determining the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area may include: based on the current coding unit being referenced multiple times, performing the following steps to determine the chroma propagation cost: for the first reference in the multiple references, determining a first amount of transferred information from the current coding unit to the reference frame at the time of the first reference in the multiple references based on the chroma intra-frame reference cost, the chroma initial propagation cost, and the chroma reference ratio, and determining a first chroma propagation cost of the current coding unit based on the chroma initial propagation cost, the first amount of transferred information, and the overlapping area; for the second reference in the multiple references, determining a first amount of transferred information from the current coding unit to the reference frame based on the chroma initial propagation cost, the first amount of transferred information, and the overlapping area. The chroma propagation cost and the chroma reference ratio are used to determine the second amount of information transferred from the current coding unit to the reference frame when the second reference is made in the multiple references, and the second chroma propagation cost of the current coding unit is determined based on the first chroma propagation cost, the second amount of information transferred, and the area of ​​the overlapping region; for subsequent references in the multiple references, the chroma propagation cost and the amount of information transferred are iteratively replaced with the chroma propagation cost and the amount of information transferred determined for the previous reference and the above steps are performed until the last chroma propagation cost of the current coding unit is determined for the last reference in the multiple references, and the chroma propagation cost is determined based on the last chroma propagation cost, so as to improve the accuracy of the chroma propagation cost by iteratively calculating the chroma propagation cost that is referenced multiple times by other coding units. Here, the chroma initial propagation cost can be set to, for example, but not limited to, 0. In addition, the chroma initial propagation cost can also be set to other values. Here, the number of iterative calculations can be based on a preset number of forward prediction frames (or coding units).

[0094] As an example, the formula To determine the chrominance propagation cost of the current coding unit that is referenced multiple times. Here, it is assumed that the 8×8 reference block may overlap with multiple 8×8 coding units in the reference frame. Here, Represents the chromaticity propagation cost after each iteration, Represents the chromaticity propagation cost before each iteration, area overlap represents the overlapping area, Indicates the amount of chromaticity information transmitted. Although the chromaticity propagation cost after each iteration in the iterative calculation process is expressed as However, after the iteration is completed, the final chromaticity propagation cost is expressed as The chroma propagation cost indicates the total amount of chroma information propagated from the current coding unit to multiple reference coding units. The larger the chroma propagation cost, the larger the total amount of chroma information propagated from the current coding unit to multiple reference frames, the more the encoding of multiple reference coding units depends on the current coding unit, and the more important the current coding unit is in the encoding process.

[0095] The luminance propagation cost of the current coding unit can be determined by a determination method that is the same as or similar to the chrominance propagation cost of the current coding unit, or by a determination method that is completely different from the chrominance propagation cost of the current coding unit. The present disclosure does not limit this.

[0096] As an example, here, the formula To determine the brightness propagation cost of the current coding unit that is referenced multiple times. Here, it is assumed that the 8×8 reference block may overlap with multiple 8×8 coding units in the reference frame. Here, Represents the brightness propagation cost after each iteration. Represents the brightness propagation cost before each iteration calculation, area overlap represents the overlapping area, Indicates the amount of information transmitted by brightness. Although the brightness propagation cost after each iteration in the iterative calculation process is expressed as However, after the iteration is completed, the final brightness propagation cost is expressed as

[0097] As an example, here, the formula To determine the amount of information transmitted by brightness here, represents the luma intra-frame reference cost, represents the brightness propagation cost, Indicates the brightness reference ratio.

[0098] As an example, here, the formula To determine the brightness reference ratio here, represents the luma intra-frame reference cost, Represents the luma inter-frame reference cost.

[0099] In step S403, a quantization parameter offset of the current coding unit is determined based on the luma intra-frame reference cost, the chroma intra-frame reference cost, and the propagation cost.

[0100] The quantization parameter offset indicates the importance of different coding units. In the coding unit tree, the larger the quantization parameter offset corresponding to a coding unit, the more the coding of other coding units depends on this coding unit, that is, the higher the degree of reference by other reference coding units, at this time, the smaller the quantization parameter corresponding to this coding unit; the smaller the quantization parameter offset corresponding to a coding unit, the less the coding of other coding units depends on this coding unit, that is, the lower the degree of reference by other reference coding units, at this time, the larger the quantization parameter corresponding to this coding unit.

[0101] In an exemplary embodiment of the present disclosure, the larger the quantization parameter offset of the current coding unit, the more the encoding of other coding units depends on the current coding unit, that is, the higher the degree of reference by other reference coding units, at this time, the smaller the quantization parameter of the current coding unit; the smaller the quantization parameter offset of the current coding unit, the less the encoding of other coding units depends on the current coding unit, that is, the lower the degree of reference by other reference coding units, at this time, the larger the quantization parameter of the current coding unit.

[0102] In an exemplary embodiment of the present disclosure, determining the quantization parameter offset of the current coding unit based on the luma intra-frame reference cost, the chroma intra-frame reference cost, and the propagation cost may include: determining the intra-frame reference cost of the current coding unit based on the luma intra-frame reference cost and the chroma intra-frame reference cost; determining the quantization parameter offset based on the intra-frame reference cost and the propagation cost, thereby improving the accuracy of the quantization parameter offset by improving the accuracy of the intra-frame reference cost.

[0103] In an exemplary embodiment of the present disclosure, determining the intra-frame reference cost of the current coding unit based on the luma intra-frame reference cost and the chroma intra-frame reference cost may include: determining the intra-frame reference cost based on a weighted calculation result of the luma intra-frame reference cost and the chroma intra-frame reference cost.

[0104] In an exemplary embodiment of the present disclosure, determining the quantization parameter offset based on the intra-frame reference cost and the propagation cost may include: determining the sum of the intra-frame reference cost and the propagation cost; determining the ratio of the sum to the intra-frame reference cost; and determining the quantization parameter offset based on the ratio.

[0105] In an exemplary embodiment of the present disclosure, determining the quantization parameter offset based on the ratio may include: determining the quantization parameter offset based on a logarithmic operation result of the ratio with a first preset value as the base.

[0106] As an example, the formula To determine the quantization parameter offset ΔQP CU Here, s represents a constant factor, represents the chroma intra-frame reference cost, C propagate represents the transmission cost, represents the luma intra-frame reference cost, and α and β represent predetermined parameters. For example, α may represent the weighting coefficient of the luma intra-frame reference cost, and β may represent the weighting coefficient of the chroma intra-frame reference cost.

[0107] In step S404, the current coding unit is quantized based on the quantization parameter offset, and the current coding unit is encoded based on the quantization process. Here, the quantization parameter corresponding to the current coding unit can be adjusted based on the quantization parameter offset, and then the current coding unit is quantized based on the adjusted quantization parameter, and the current coding unit is encoded based on the quantization process.

[0108] In the quantization process, the residual image (or residual) will be quantized through the quantization step, and the quantization step is generally determined based on the quantization parameter. The quantization parameter reflects the compression of spatial details. The quantization parameter value determines the quality of the encoded video to a certain extent. Generally, the smaller the value of the quantization parameter, the more details will be retained and the clarity of the video will be higher. The larger the value of the quantization parameter, the more details will be discarded and the clarity of the video will be lower.

[0109] In an exemplary embodiment of the present disclosure, encoding the current coding unit based on the quantization process may include: determining a chroma quantization parameter offset and a luminance quantization parameter offset of the current coding unit; determining a chroma encoding rate of the current coding unit based on the chroma quantization parameter offset and the luminance quantization parameter offset; encoding the current coding unit based on the quantization process and the chroma encoding rate, thereby improving encoding performance by adaptively determining the chroma encoding rate of the current coding unit by utilizing the chroma quantization parameter offset and the luminance quantization parameter offset.

[0110] In an exemplary embodiment of the present disclosure, determining the chroma encoding rate of the current coding unit based on the chroma quantization parameter offset and the luminance quantization parameter offset may include: determining a rate-distortion cost function related to the chroma encoding rate based on the chroma quantization parameter offset and the luminance quantization parameter offset; determining the chroma encoding rate based on the rate-distortion cost function, thereby improving the accuracy of the chroma encoding rate of the current coding unit by adaptively adjusting the rate-distortion cost function by utilizing the offsets of the chroma component and the luminance component.

[0111] As an example, the chroma coding rate can be determined by the rate-distortion cost function J=D+λR. Here, J represents the rate-distortion cost, D represents the distortion, R represents the chroma coding rate, and λ is the Lagrange multiplier. to determine the Lagrange multiplier. Here, Indicates the chromaticity quantization parameter offset, represents the brightness quantization parameter offset, λ represents the Lagrange multiplier before calculation, λ′ represents the Lagrange multiplier after calculation, and x represents a predetermined calculation parameter.

[0112] In an exemplary embodiment of the present disclosure, determining the chroma quantization parameter offset of the current coding unit may include: determining the chroma quantization parameter offset based on the chroma intra reference cost and the chroma propagation cost.

[0113] In an exemplary embodiment of the present disclosure, determining the chroma quantization parameter offset based on the intra-chroma reference cost and the chroma propagation cost may include: determining the chroma cost sum of the intra-chroma reference cost and the chroma propagation cost; determining the chroma cost ratio of the chroma cost sum to the intra-chroma reference cost; and determining the chroma quantization parameter offset based on the chroma cost ratio.

[0114] In an exemplary embodiment of the present disclosure, determining the chromaticity quantization parameter offset based on the chromaticity cost ratio may include: determining the chromaticity quantization parameter offset based on a logarithmic operation result of the chromaticity cost ratio with a second preset value as the base.

[0115] As an example, the formula To calculate the color quantization parameter offset Here, s represents a constant factor, represents the chroma intra reference cost, represents the chroma propagation cost.

[0116] As an example, the formula To calculate the brightness quantization parameter offset Here, s represents a constant factor, represents the luma intra-frame reference cost, represents the brightness propagation cost.

[0117] The above has been combined Figure 4 A video encoding method according to an exemplary embodiment of the present disclosure is described. Figure 5 A video encoding apparatus and units thereof according to an exemplary embodiment of the present disclosure are described.

[0118] Figure 5 A block diagram of a video encoding apparatus according to an exemplary embodiment of the present disclosure is shown.

[0119] Reference Figure 5The video encoding device includes a reference cost determination unit 51, a propagation cost determination unit 52, an offset determination unit 53 and an encoding unit 54.

[0120] The reference cost determination unit 51 is configured to determine the luma intra-frame reference cost and the luma inter-frame reference cost, the chroma intra-frame reference cost and the chroma inter-frame reference cost of the current coding unit.

[0121] The propagation cost determination unit 52 is configured to determine the propagation cost of the current coding unit based on the luma intra reference cost and the luma inter reference cost and the chroma intra reference cost and the chroma inter reference cost.

[0122] In an exemplary embodiment of the present disclosure, the propagation cost determination unit 52 may be configured to: determine the luminance propagation cost of the current coding unit based on the luminance intra-frame reference cost and the luminance inter-frame reference cost; determine the chrominance propagation cost of the current coding unit based on the chrominance intra-frame reference cost and the chrominance inter-frame reference cost; determine the propagation cost based on the luminance propagation cost and the chrominance propagation cost.

[0123] In an exemplary embodiment of the present disclosure, the propagation cost determining unit 52 may be configured to determine the propagation cost based on a weighted calculation result of the luminance propagation cost and the chrominance propagation cost.

[0124] In an exemplary embodiment of the present disclosure, the propagation cost determination unit 52 may be configured to: determine the chroma reference ratio passed from the current coding unit to the reference frame based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; determine the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area of ​​the current coding unit and the reference frame.

[0125] In an exemplary embodiment of the present disclosure, the propagation cost determination unit 52 may be configured to: determine the amount of chrominance information transmitted from the current coding unit to the reference frame based on the chrominance intra-frame reference cost, the chrominance initial propagation cost of the current coding unit, and the chrominance reference ratio; determine the chrominance propagation cost based on the chrominance initial propagation cost, the amount of transmitted information, and the overlapping region area.

[0126] In an exemplary embodiment of the present disclosure, the propagation cost determination unit 52 may be configured to: based on the current coding unit being referenced multiple times, perform the following steps to determine the chroma propagation cost: for the first reference in the multiple references, determine the first amount of information transferred from the current coding unit to the reference frame at the time of the first reference in the multiple references based on the chroma intra-frame reference cost, the chroma initial propagation cost, and the chroma reference ratio; determine the first chroma propagation cost of the current coding unit based on the chroma initial propagation cost, the first amount of information transferred, and the overlapping area; for the second reference in the multiple references, determine the first amount of information transferred from the current coding unit to the reference frame based on the chroma intra-frame reference cost, the first chroma initial propagation cost, and the overlapping area. propagation cost and the chrominance reference ratio, determine the second amount of transferred information transmitted from the current coding unit to the reference frame when the second reference is made in the multiple references, and determine the second chrominance propagation cost of the current coding unit based on the first chrominance propagation cost, the second amount of transferred information, and the area of ​​the overlapping region; for subsequent references in the multiple references, iteratively replace the chrominance propagation cost and the amount of transferred information with the chrominance propagation cost and the amount of transferred information determined for the previous reference and perform the above steps until the last chrominance propagation cost of the current coding unit is determined for the last reference in the multiple references, and determine the chrominance propagation cost based on the last chrominance propagation cost.

[0127] The offset determination unit 53 is configured to determine a quantization parameter offset of the current coding unit based on the luma intra-frame reference cost, the chroma intra-frame reference cost, and the propagation cost.

[0128] In an exemplary embodiment of the present disclosure, the offset determination unit 53 may be configured to: determine the intra-frame reference cost of the current coding unit based on the luma intra-frame reference cost and the chroma intra-frame reference cost; and determine the quantization parameter offset based on the intra-frame reference cost and the propagation cost.

[0129] The encoding unit 54 is configured to perform a quantization process on the current coding unit based on the quantization parameter offset, and encode the current coding unit based on the quantization process.

[0130] In an exemplary embodiment of the present disclosure, the encoding unit 54 may be configured to: determine a chroma quantization parameter offset and a luminance quantization parameter offset of the current coding unit; determine a chroma encoding rate of the current coding unit based on the chroma quantization parameter offset and the luminance quantization parameter offset; and encode the current coding unit based on the quantization process and the chroma encoding rate.

[0131] In an exemplary embodiment of the present disclosure, the encoding unit 54 may be configured to: determine a rate-distortion cost function related to the chroma encoding rate based on the chroma quantization parameter offset and the luminance quantization parameter offset; and determine the chroma encoding rate based on the rate-distortion cost function.

[0132] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] The above has been combined Figure 5 The video encoding device according to the exemplary embodiment of the present disclosure is described. Figure 6 A computing environment coupled with a user interface according to some embodiments of the present disclosure is described.

[0134] Figure 6 A computing environment 610 is shown coupled to a user interface 650. The computing environment 610 may be part of a data processing server. The computing environment 610 includes a processor 620, a memory 630, and an input / output (I / O) interface 640.

[0135] The processor 620 generally controls the overall operation of the computing environment 610, such as operations associated with display, data acquisition, data communication, and image processing. The processor 620 may include one or more processors for executing instructions to perform all or some of the steps in the above method. In addition, the processor 620 may include one or more modules that facilitate the interaction between the processor 620 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0136] The memory 630 is configured to store various types of data to support the operation of the computing environment 610. The memory 630 may include predetermined software 632. Examples of such data include instructions for any application or method operating on the computing environment 610, video data sets, image data, etc. The memory 630 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0137] I / O interface 640 provides an interface between processor 620 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 640 may be coupled to an encoder and a decoder.

[0138] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 630 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 620 in a computing environment 610 to perform the above method. In one example, the plurality of programs can be executed by a processor 620 in a computing environment 610 to (for example, from Figure 2 The video encoder 20 in the computing environment 610 receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 620 in the computing environment 610 to perform the above-mentioned decoding method according to the received bitstream or data stream. In another example, the multiple programs can be executed by the processor 620 in the computing environment 610 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 620 in the computing environment 610 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0139] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0140] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 620); and a non-transitory computer-readable storage medium or memory 630 in which multiple programs that can be executed by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0141] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes the encoded video information generated by the above encoding method or the encoded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 630 is also provided, wherein the plurality of programs can be executed by a processor 620 in a computing environment 610 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0142] In an embodiment, the computing environment 610 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0143] The above has been referred to Figure 4 and Figure 5 The video encoding method and apparatus according to the exemplary embodiments of the present disclosure are described. However, it should be understood that: Figure 5 The video encoding device and its units shown in the figure may be respectively configured as software, hardware, firmware or any combination of the above items to perform specific functions. Figure 6 The computing environment shown in is not limited to including the components shown above, but some components may be added or deleted as needed, and the above components may also be combined.

[0144] According to the video encoding method and device disclosed in the present invention, by determining the quantization parameter offset of the current coding unit based on the luma frame intra-reference cost, the chroma frame intra-reference cost, and the propagation cost based on the chroma frame intra-reference cost and the chroma frame inter-reference cost, the accuracy of the quantization parameter offset of the current coding unit can be improved, and the current coding unit is quantized based on the highly accurate quantization parameter offset, and the current coding unit is encoded based on the quantization process, thereby improving the encoding performance.

[0145] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0146] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0147] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video encoding method, characterized in that: include: Determine a luma intra-frame reference cost and a luma inter-frame reference cost, a chroma intra-frame reference cost and a chroma inter-frame reference cost of a current coding unit; Determine a propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost and the chroma intra-frame reference cost and the chroma inter-frame reference cost; Determine a quantization parameter offset of the current coding unit based on the luma intra-frame reference cost, the chroma intra-frame reference cost, and the propagation cost; The current coding unit is quantized based on the quantization parameter offset, and the current coding unit is encoded based on the quantization.

2. The video encoding method according to claim 1, characterized in that The determining the propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost and the chroma intra-frame reference cost and the chroma inter-frame reference cost includes: Determine a luma propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost; Determine a chroma propagation cost of the current coding unit based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; The propagation cost is determined based on the luma propagation cost and the chroma propagation cost.

3. The video encoding method according to claim 2, characterized in that: The determining the propagation cost based on the luminance propagation cost and the chrominance propagation cost comprises: The propagation cost is determined based on a weighted calculation result of the brightness propagation cost and the chrominance propagation cost.

4. The video encoding method according to claim 2, characterized in that: The determining the chroma propagation cost of the current coding unit based on the chroma intra-frame reference cost and the chroma inter-frame reference cost includes: Determining a chroma reference ratio value transferred from the current coding unit to a reference frame based on the chroma intra-frame reference cost and the chroma inter-frame reference cost; The chroma propagation cost is determined based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area of ​​the current coding unit and the reference frame.

5. The video encoding method according to claim 4, characterized in that: The determining the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area of ​​the current coding unit and the reference frame includes: Determining an amount of transferred chroma information transferred from the current coding unit to a reference frame based on the chroma intra-frame reference cost, the chroma initial propagation cost of the current coding unit, and the chroma reference ratio; The chrominance propagation cost is determined based on the chrominance initial propagation cost, the amount of transmitted information, and the area of ​​the overlapping region.

6. The video encoding method according to claim 4, characterized in that: The determining the chroma propagation cost based on the chroma intra-frame reference cost, the chroma reference ratio, the chroma initial propagation cost of the current coding unit, and the overlapping area includes: Based on the current coding unit being referenced multiple times, the following steps are performed to determine the chroma propagation cost: For a first reference among the multiple references, based on the chroma intra-frame reference cost, the chroma initial propagation cost, and the chroma reference ratio, determine a first transfer information amount transferred from the current coding unit to the reference frame when the first reference among the multiple references is made, and determine a first chroma propagation cost of the current coding unit based on the chroma initial propagation cost, the first transfer information amount, and the overlapping area; For a second reference in the multiple references, based on the chroma intra-frame reference cost, the first chroma propagation cost, and the chroma reference ratio, determine a second transfer information amount transferred from the current coding unit to the reference frame when the second reference in the multiple references is used, and determine a second chroma propagation cost of the current coding unit based on the first chroma propagation cost, the second transfer information amount, and the overlapping area; For subsequent references in the multiple references, iteratively replace the chromaticity propagation cost and the amount of transmitted information with the chromaticity propagation cost and the amount of transmitted information determined for the previous reference and perform the above steps until the last chromaticity propagation cost of the current coding unit is determined for the last reference in the multiple references, and the chromaticity propagation cost is determined based on the last chromaticity propagation cost.

7. The video encoding method according to claim 1, characterized in that: The determining the quantization parameter offset of the current coding unit based on the luma frame intra reference cost, the chroma frame intra reference cost, and the propagation cost includes: Determine an intra-frame reference cost of the current coding unit based on the luma intra-frame reference cost and the chroma intra-frame reference cost; The quantization parameter offset is determined based on the intra reference cost and the propagation cost.

8. The video encoding method according to claim 1, characterized in that: The encoding of the current coding unit based on the quantization process includes: Determining a chrominance quantization parameter offset and a luminance quantization parameter offset of the current coding unit; Determine a chroma encoding bit rate of the current coding unit based on the chroma quantization parameter offset and the luma quantization parameter offset; The current coding unit is encoded based on the quantization process and the chroma encoding rate.

9. The video encoding method according to claim 8, characterized in that: The determining the chroma encoding rate of the current coding unit based on the chroma quantization parameter offset and the luminance quantization parameter offset includes: Determining a rate-distortion cost function related to the chroma encoding bit rate based on the chroma quantization parameter offset and the luma quantization parameter offset; The chroma encoding rate is determined based on the rate-distortion cost function.

10. A video encoding device, characterized in that: include: A reference cost determination unit, configured to determine a luma intra-frame reference cost and a luma inter-frame reference cost, a chroma intra-frame reference cost and a chroma inter-frame reference cost of a current coding unit; A propagation cost determining unit, configured to determine a propagation cost of the current coding unit based on the luma intra-frame reference cost and the luma inter-frame reference cost and the chroma intra-frame reference cost and the chroma inter-frame reference cost; an offset determining unit, configured to determine a quantization parameter offset of the current coding unit based on the luma intra-frame reference cost, the chroma intra-frame reference cost, and the propagation cost; and The encoding unit is configured to perform a quantization process on the current encoding unit based on the quantization parameter offset, and encode the current encoding unit based on the quantization process.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video encoding method according to any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor of an electronic device, the electronic device is enabled to perform the video encoding method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video encoding method according to any one of claims 1 to 9 is implemented.

Images