Image encoding method and image encoding device

Abstract

The invention provides an image encoding method and an image encoding device. The image coding method comprises the steps that first quantization parameter information of a code rate control unit block in an image to be filtered is acquired, and the code rate control unit block represents a processing unit used for code rate distribution and image quality control; adjusting a filtering weight of a filtering block to be subjected to motion compensation time domain filtering in the bit rate control unit block based on the first quantization parameter information; performing motion compensation time domain filtering on the filtering block based on the adjusted filtering weight; and performing encoding based on the filtering result.

Description

Technical Field

[0001] This disclosure relates to video encoding / decoding and compression. More specifically, this disclosure relates to an image encoding method, an image encoding apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Various electronic devices support digital video. These devices transmit and receive, or otherwise transfer, digital video data via communication networks, and / or store digital video data on storage devices. Because communication networks have limited bandwidth capacity and storage devices have limited storage resources, video data can be compressed using one or more video codec standards before transmission or storage to generate coded video data using a lower bit rate, while avoiding or minimizing video quality degradation. Summary of the Invention

[0003] Embodiments of this disclosure provide an image encoding method, an image encoding apparatus, an electronic device, a computer-readable storage medium, and a computer program product that utilize quantization parameter (QP) information derived from a rate control (RC) module to adjust the filtering weights of the filter blocks to be subjected to motion-compensated temporal filtering (MCTF). This addresses the problem of QP errors caused by inconsistencies between the video used for rate control and the actual encoded video due to the introduction of MCTF, thereby improving video encoding and decoding performance.

[0004] According to a first aspect of this disclosure, an image encoding method is provided, the method comprising: acquiring first quantization parameter information of a bitrate control block in an image to be filtered, wherein the bitrate control block represents a processing unit for bitrate allocation and image quality control; adjusting the filtering weights of a filtering block in the bitrate control block to be subjected to motion-compensated temporal filtering based on the first quantization parameter information; performing motion-compensated temporal filtering on the filtering block based on the adjusted filtering weights; and performing encoding based on the filtering result.

[0005] Optionally, adjusting the filter weights of the filter blocks in the bitrate control block that will undergo motion compensation temporal filtering based on the first quantization parameter information includes: determining a weight adjustment factor for adjusting the filter weights based on the first quantization parameter information; and adjusting the filter weights of the filter blocks based on the weight adjustment factor.

[0006] Optionally, determining the weight adjustment factor for the filter block based on the first quantization parameter information includes: obtaining second quantization parameter information of the coding tree unit including the bit rate control unit block; and determining the weight adjustment factor based on the first quantization parameter information and the second quantization parameter information.

[0007] Optionally, obtaining the second quantization parameter information of the coding tree unit including the bit rate control block includes: obtaining the first quantization parameter information of each bit rate control block included in the coding tree unit; and determining the second quantization parameter information of the coding tree unit based on the first quantization parameter information of each bit rate control block.

[0008] Optionally, determining the second quantization parameter information of the coding tree unit based on the first quantization parameter information of each bitrate control block includes: summing and averaging the first quantization parameter information of each bitrate control block to obtain the second quantization parameter information of the coding tree unit.

[0009] Optionally, determining the weight adjustment factor based on the first quantization parameter information and the second quantization parameter information includes: determining the ratio between the first quantization parameter information and the second quantization parameter information; comparing the ratio with a first threshold and a second threshold to determine the weight adjustment factor, wherein the first threshold is less than or equal to the second threshold.

[0010] Optionally, comparing the ratio with a first threshold and a second threshold to determine the weight adjustment factor includes: setting the weight adjustment factor to a preset value when the ratio is less than the first threshold; setting the weight adjustment factor to the second threshold when the ratio is greater than the second threshold; and setting the weight adjustment factor to a value equal to the ratio when the ratio is greater than or equal to the first threshold and less than or equal to the second threshold.

[0011] Optionally, the first threshold and the second threshold are determined based on the frame type of the image to be filtered.

[0012] Optionally, adjusting the filtering weights of the filtering blocks in the bitrate control block that will undergo motion-compensated temporal filtering based on the first quantization parameter information includes: obtaining the second quantization parameter information of each coding tree unit in the image to be filtered; determining the number of coding tree units in the image to be filtered whose second quantization parameter information is greater than the second quantization parameter information of the coding tree units including the bitrate control block; determining the ratio between the number and the total number of coding tree units in the image to be filtered; and adjusting the filtering weights of the filtering blocks if the ratio is greater than a preset threshold.

[0013] Optionally, the method further includes: skipping the adjustment of the filtering weights of the filter block if the ratio is less than or equal to the preset threshold.

[0014] According to a second aspect of this disclosure, an image encoding apparatus is provided, the apparatus comprising: an acquisition module configured to: acquire first quantization parameter information of a bitrate control block in an image to be filtered, wherein the bitrate control block represents a processing unit for bitrate allocation and image quality control; a filtering module configured to: adjust the filtering weights of a filtering block in the bitrate control block to be subjected to motion-compensated temporal filtering based on the first quantization parameter information, and perform motion-compensated temporal filtering on the filtering block based on the adjusted filtering weights; and an encoding module configured to: encode the motion-compensated temporal filtering filtering block based on the first quantization parameter information.

[0015] Optionally, the filtering module is configured to: determine a weight adjustment factor for adjusting the filtering weights based on the first quantization parameter information; and adjust the filtering weights of the filtering block based on the weight adjustment factor.

[0016] Optionally, the filtering module is configured to: obtain second quantization parameter information of the coding tree unit including the bit rate control unit block; and determine the weight adjustment factor based on the first quantization parameter information and the second quantization parameter information.

[0017] Optionally, the filtering module is configured to: obtain first quantization parameter information of each rate control block included in the coding tree unit; and determine second quantization parameter information of the coding tree unit based on the first quantization parameter information of each rate control block.

[0018] Optionally, the filtering module is configured to: sum and then average the first quantization parameter information of each bit rate control unit block to obtain the second quantization parameter information of the coding tree unit.

[0019] Optionally, the filtering module is configured to: determine the ratio between the first quantization parameter information and the second quantization parameter information; compare the ratio with a first threshold and a second threshold to determine the weight adjustment factor, wherein the first threshold is less than or equal to the second threshold.

[0020] Optionally, the filtering module is configured to: set the weight adjustment factor to a preset value when the ratio is less than the first threshold; set the weight adjustment factor to the second threshold when the ratio is greater than the second threshold; and set the weight adjustment factor to a value equal to the ratio when the ratio is greater than or equal to the first threshold and less than or equal to the second threshold.

[0021] Optionally, the first threshold and the second threshold are determined based on the frame type of the image to be filtered.

[0022] Optionally, the apparatus further includes: a determining module configured to: acquire second quantization parameter information of each coding tree unit in the image to be filtered; determine the number of coding tree units in the image to be filtered whose second quantization parameter information is greater than the second quantization parameter information of the coding tree unit including the bitrate control unit block; determine the ratio between the number and the total number of coding tree units in the image to be filtered; and, if the ratio is greater than a preset threshold, determine to adjust the filtering weight of the filtering block.

[0023] Optionally, the determining module is configured to: if the ratio is less than or equal to the preset threshold, determine to skip the adjustment of the filtering weights of the filtering block.

[0024] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the image encoding method of this disclosure.

[0025] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores a bit stream and instructions, wherein the instructions, when executed by a computing device having one or more processors, cause the one or more processors to perform an image encoding method of this disclosure to generate the bit stream.

[0026] According to a fifth aspect of this disclosure, a computer program product is provided, wherein instructions in the computer program product, when executed by a computing device having one or more processors, cause the one or more processors to perform the image encoding method of this disclosure to generate a bitstream.

[0027] According to a sixth aspect of this disclosure, a method for storing a bitstream is provided, comprising: performing an image encoding method of this disclosure to generate a bitstream; and storing the bitstream.

[0028] It will be understood that the above general description and the following detailed description are merely examples and do not limit this disclosure. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate examples according to this disclosure and, together with this description, serve to explain the principles of this disclosure.

[0030] Figure 1 This is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0031] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0032] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0033] Figures 4A to 4E This is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of this disclosure.

[0034] Figure 5 This is a schematic diagram illustrating a video coding framework based on bitrate control and MCTF according to some embodiments of the present disclosure.

[0035] Figure 6 This is a schematic diagram illustrating a GOP encoding structure according to some embodiments of the present disclosure.

[0036] Figure 7 A schematic diagram of an MCTF according to some embodiments of the present disclosure is shown.

[0037] Figure 8 A schematic diagram of a bit rate control unit block and an MCTF filter block according to some embodiments of the present disclosure is shown.

[0038] Figure 9 This is a flowchart illustrating an image encoding method according to some embodiments of the present disclosure.

[0039] Figure 10 This is a flowchart illustrating another image encoding apparatus according to some embodiments of the present disclosure.

[0040] Figure 11 This is a block diagram illustrating an image encoding apparatus according to some embodiments of the present disclosure.

[0041] Figure 12 This is a diagram illustrating a computing environment coupled to a user interface according to some embodiments of the present disclosure. Detailed Implementation

[0042] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. Numerous non-limiting details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0043] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this disclosure are used to distinguish objects and not to describe any specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in sequences other than those shown in the drawings or described in this disclosure.

[0044] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data for later decoding by a target device 14. The source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0045] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0046] Video, whether captured, pre-captured, or computer-generated, can be encoded by video encoder 20. Encoded video data may include a series of images, each of which may include one or more sample arrays, for example, for monochrome, only luminance (Y); luminance and two chrominances in the YCbCr or YCgCo domain; or green, blue, and red in the GBR (also known as RGB) domain. For ease of reference and terminology in this application, in some embodiments, the variables and terms associated with each set having three sample arrays may be referred to as luminance and chrominance, where the two chrominance arrays may be referred to as Cb and Cr, regardless of the actual color representation used. Video data may be in chrominance format 4:0:0, chrominance format 4:2:0, chrominance format 4:2:2, or chrominance format 4:4:4, but this application is not limited thereto. The bit depth (BitDepth) of the sample array can be an integer ranging from 8 to 16, for example, a value of BitDepth of 8, 9, 10, 11, 12, 13, 14, 15, or 16. It should be noted that the value of BitDepth is not limited to this, but can be any other value proposed in the future.

[0047] Encoded video data can be directly transmitted to the target device 14 via link 16 through the output interface 22 of the source device 12. Output interface 22 may include a modem and / or a transmitter. Link 16 may include any type of wireless communication medium or device and / or any type of wired communication medium or device capable of transmitting encoded video data from the source device 12 to the target device 14. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by the target device 14 or by other devices via, for example, input interface 28, for decoding and / or playback. Storage device 32 may include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital universal discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0048] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. Alternatively, target device 14 may access stored video data from storage device 32 via streaming, downloading, or a combination of both. Encoded video data may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Display device 34 may be an integrated display device or an external display device configured to communicate with target device 14 and may display the decoded video data to a user.

[0049] The video encoder 20 and video decoder 30 can operate (e.g., encode and decode video data) according to proprietary or industry standards (e.g., Universal Video Codec (VVC), Joint Exploratory Test Model (JEM), High Efficiency Video Codec (HEVC / H.265), Advanced Video Codec (AVC / H.264), Moving Picture Experts Group (MPEG) codec) or extensions of such standards. It should be understood that this application is not limited to any specific video coding / decoding standard and may be applicable to other current and future video coding / decoding standards.

[0050] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0051] In some implementations, components of source device 12 and / or target device 14 (e.g., Figure 1 , Figure 2 and / or Figure 3At least some of the components shown herein can operate in a cloud computing service network such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS), where the cloud computing service network can provide software, platform, and / or infrastructure. In some embodiments, one or more components of source device 12 and / or target device 14 not included in the cloud computing service network can be located in one or more client devices, and the one or more client devices can communicate with server computers in the cloud computing service network via wireless communication networks or wired communication networks. In one embodiment, at least some of the operations described herein can be implemented as cloud-based services provided by one or more server computers, wherein the one or more server computers are implemented by at least some of the components of source device 12 and / or target device 14 in the cloud computing service network; and one or more other operations described herein can be implemented by one or more client devices. In some embodiments, the cloud computing service network can be a private cloud, a public cloud, or a hybrid cloud. Without departing from the scope of this disclosure, terms such as “cloud,” “cloud computing,” and “cloud-based” are used interchangeably as appropriate. It should be understood that this disclosure is not limited to implementation in the aforementioned cloud computing service networks. Instead, this disclosure can also be implemented in any other type of computing environment currently known or to be developed in the future.

[0052] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application.

[0053] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (IBC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63, such as a deblocking filter, can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can be used to filter the output of the adder 62. It should be noted that this application is not limited to the embodiments described herein regarding CCSAO technology. It can also be applied to situations where one or more samples of any one of the luminance component and two chrominance components (as mentioned above, for ease of reference and terminology in this application, in the YCbCr domain, the luminance component and the two chrominance components can represent Y, Cb, and Cr; in the YCgCo domain, the luminance component and the two chrominance components can represent Y, Cg, and Co; or in the RGB domain, the luminance component and the two chrominance components can represent G, B, and R) are used to select an offset for the sample points of any other component among the luminance component and the two chrominance components, in order to modify the sample points of that other component based on the selected offset. Alternatively, a SAO technology is also provided, which is substantially the same as CCSAO technology, except that in this SAO technology, an offset is selected for the sample points of any one of the luminance component and the two chrominance components based on one or more samples of that one component, in order to modify the sample points of that one component based on the selected offset. Furthermore, it should be noted that the first component mentioned herein can be either the luminance component or either of the two chrominance components, the second component mentioned herein can be either the luminance component or either of the two chrominance components, and the third component mentioned herein can be the remaining component of the luminance component and the two chrominance components. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of a fixed or programmable hardware unit, or it can be distributed among one or more of the fixed or programmable hardware units described.

[0054] The video data storage device 40 can store video data encoded by the components of the video encoder 20. For example, it can store data from... Figure 1The video source 18 shown receives video data from the video data memory 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) used by the video encoder 20 when encoding the video data. The video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, the video data memory 40 may be on-chip along with other components of the video encoder 20, or off-chip relative to those components. It should be noted that in the field of video encoding and decoding, the term "frame" can be used as a synonym for the terms "image" or "picture".

[0055] like Figure 2 As shown, after receiving video data, segmentation unit 45 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a collection of video blocks) or other larger coding units (CUs) according to a predefined splitting structure associated with the video data (e.g., a quadtree (QT) structure). A video frame is, or can be considered, a two-dimensional array or matrix of sample points with sample values. Sample points in the array may also be referred to as pixels or image elements (pel). The number of sample points in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the video frame. For example, a video frame can be divided into multiple video blocks by using QT segmentation. A video block is again, or can be considered, a two-dimensional array or matrix of sample points with sample values, but its dimension is smaller than that of the video frame. The number of sample points in the horizontal and vertical directions (or axes) of the video block defines the size of the video block. By iteratively using, for example, QT segmentation, binary tree (BT) segmentation, or ternary tree (TT) segmentation, or any combination thereof, a video block can be further segmented into one or more block partitions or sub-blocks (which can again form blocks). It should be noted that the term "block" or "video block" as used herein can refer to a portion of a frame or image, particularly a rectangular (square or non-square) portion. Referring to, for example, HEVC and VVC, a block or video block can be or corresponds to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU) and / or can be or corresponds to a corresponding block (e.g., coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB)) and / or sub-block.

[0056] Prediction processing unit 41 can select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one of several intra-frame or inter-frame predictive coding modes. Prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coded block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coded block for subsequent use as part of a reference frame. Prediction processing unit 41 also provides at least one of the syntax elements (e.g., motion vectors, intra-frame or inter-frame mode indicators, segmentation information, and other such syntax information) to entropy coding unit 56.

[0057] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 may perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and the motion compensation unit 44 perform inter-predictive coding of the current video block relative to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 may perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0058] In some implementations, motion estimation unit 42 generates a motion vector for the current block according to a predetermined pattern within the video frame sequence during motion estimation. The motion vector can indicate the displacement of a video block within the current frame relative to a predicted block within a reference frame associated with the current block being encoded. The predetermined pattern can designate video frames in the sequence as P-frames or B-frames. In some implementations, the motion vector prediction value (MVP) of the current block is subtracted from the actual motion vector of the current block to produce the motion vector difference (MVD) of the current block, where the MVP can be determined based on motion information from spatially adjacent blocks and / or temporally co-located blocks of the current block. The MVP and MVD information can then be encoded into the video bitstream instead of the actual motion vector of the current block. IBC unit 48 can determine the vectors (e.g., block vectors) for IBC encoding / decoding in a similar manner to how motion estimation unit 42 determines the motion vectors for inter-frame prediction, or the block vectors can be determined using motion estimation unit 42. It should be noted that the IBC pattern can be considered an intra-frame prediction pattern, or a prediction pattern other than intra-frame and inter-frame prediction patterns.

[0059] Regarding pixel differences, the predicted block for a video block can be, or can correspond to, a block or reference block of a reference frame considered to closely match the video block to be encoded. Pixel differences can be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional pixel accuracy.

[0060] The motion estimation unit 42 determines the motion vector information of a video block in an inter-frame predictive coding frame by comparing the position of the video block with the position of the predicted block of a reference frame selected from either a first reference frame list (list 0) or a second reference frame list (list 1), where each reference frame list identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the determined motion vector information to the motion compensation unit 44, and then to the entropy coding unit 56.

[0061] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vector information determined by motion estimation unit 42. Upon receiving motion vector information for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of the video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0062] In some implementations, IBC unit 48 may generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with motion estimation unit 42 and motion compensation unit 44, but these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors.

[0063] In other examples, IBC unit 48 may use motion estimation unit 42 and motion compensation unit 44, in whole or in part, to perform such functions for IBC prediction according to the embodiments described herein. In any case, for intra-block copying, in terms of pixel difference, the predicted block may be a block considered to closely match the block to be encoded, the pixel difference may be determined by SAD, SSD, or other difference metrics, and identifying the predicted block may include calculating values ​​for sub-integer pixel positions.

[0064] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the IBC unit 48, the intra-frame prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-frame prediction processing unit 46 can determine the intra-frame prediction mode for encoding the current block. The intra-frame prediction processing unit 46 can provide information indicating the selected intra-frame prediction mode for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0065] After prediction processing unit 41 determines the prediction block for the current video block, adder 50 forms a residual block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block (forming a pixel difference). The pixel difference may include a luminance component difference, a chrominance component difference, or both. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using one or more transforms (e.g., Discrete Cosine Transform (DCT) or a conceptually similar transform).

[0066] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can subsequently perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0067] After quantization, the entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to, for example,... Figure 1 The video decoder 30 shown, or archived in, for example Figure 1The data is stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy code the motion vectors and other syntax elements used for the current video frame.

[0068] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain for generating a reference block for predicting other video blocks.

[0069] Adder 62 adds the reconstructed residual block to the predicted block to produce a reference block to be stored in DPB 64.

[0070] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 includes a motion compensation unit 82, an intra-frame prediction unit 84, and an IBC unit 85. The video decoder 30 can perform operations in conjunction with the above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on the motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80.

[0071] In some examples, the units of video decoder 30 may be assigned tasks to perform embodiments of the present disclosure. Furthermore, in some examples, embodiments of the present disclosure may be distributed across one or more units of video decoder 30.

[0072] Video data memory 79 can store video data, such as encoded video bitstreams, that will be decoded by other components of video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. The DPB 92 of video decoder 30 stores reference video data for use by video decoder 30 when decoding video data. Video data memory 79 and DPB 92 can be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. In some examples, video data memory 79 may be on-chip along with other components of video decoder 30, or off-chip relative to those components.

[0073] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 performs entropy decoding on the bitstream to generate quantization coefficients, motion vector information or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0074] When a video frame is encoded as an intra-predictive coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-predictive unit 84 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by signaling and reference data from the previous decoded block of the current frame.

[0075] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 generates one or more prediction blocks for the current video frame based on motion vector information and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0076] In some examples, when encoding a video block according to the IBC mode described herein, IBC unit 85 generates a predicted block for the current video block based on block vector information and other syntax elements received from entropy decoding unit 80. The predicted block can be located within a reconstructed region of the same image as the current video block, defined by video encoder 20.

[0077] Motion compensation unit 82 and / or IBC unit 85 determine prediction information for video blocks in the current video frame by parsing vector information and other syntax elements, and then use the prediction information to generate prediction blocks for the current video blocks. For example, motion compensation unit 82 uses some syntax elements from the received syntax elements to determine the prediction mode for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0078] Similarly, IBC unit 85 can use some of the syntax elements in the received syntax elements, such as flags, to determine which video blocks in the current video frame are predicted using the IBC mode, which video blocks in the frame are in the reconstruction region and should be stored in the DPB 92, the block vector for each IBC predicted video block in the frame, the IBC prediction state for each IBC predicted video block in the frame, and other information for decoding the video blocks in the current video frame.

[0079] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 based on the received syntax elements and use these interpolation filters to generate the prediction block.

[0080] Similar to the process of selecting a prediction block in a reference frame during inter-frame prediction of a video block, both the video encoder 20 and the video decoder 30 need to employ a set of rules to construct a motion vector candidate list (also known as a "merging list") for the current block using those potential candidate motion vectors associated with spatially neighboring and / or temporally co-located blocks. Then, a member is selected from the motion vector candidate list as the motion vector prediction value for the current block. By doing so, it is not necessary to send the motion vector candidate list itself from the video encoder 20 to the video decoder 30, and the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for both the video encoder 20 and the video decoder 30 to encode and decode the current block using the same motion vector prediction value from the motion vector candidate list.

[0081] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0082] Adder 90 reconstructs the decoded video block for the current video block by adding the residual block from inverse transform processing unit 88 to the corresponding prediction block. A loop filter 91 (e.g., a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be located between adder 90 and DPB 92 for further processing of the decoded video block. In some examples, loop filter 91 may be omitted, and the decoded video block may be directly provided to DPB 92 by adder 90. The decoded video block in a given frame is then stored in DPB 92, which stores reference frames for subsequent motion compensation of the next video block. DPB 92, or a separate memory device from DPB 92, may also store the decoded video for later presentation on a display device (e.g., ...). Figure 1 On the display device 34).

[0083] In a typical video encoding and decoding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of chroma samples (Cb). SCr is a two-dimensional array of chroma samples (Cr). In other instances, a frame may be monochrome and therefore consist of only a two-dimensional array of luma samples.

[0084] like Figure 4AAs shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of CTUs. A video frame may include an integer number of CTUs ordered consecutively from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. Figure 4B As shown, each CTU may include a CTB for luma samples, two corresponding CTBs for chroma samples, and syntax elements for encoding the samples of the CTBs. The syntax elements describe the properties of different types of units of the encoded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and / or other parameters. In a monochrome image or an image with three separate color planes, the CTU may include a single CTB and syntax elements for encoding the samples of that CTB. The CTB may be an N×N sample block.

[0085] To achieve better performance, the video encoder 20 can recursively perform tree segmentation on the CTU's CTB, such as binary tree segmentation, ternary tree segmentation, quadtree segmentation, or a combination thereof, and divide the CTU into smaller CUs. For example... Figure 4C As described, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Of these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, CU 430 and CU 440, are further divided into four CUs with a block size of 8×8. Figure 4D Depicting as shown Figure 4C The final result of the CTU 400 partitioning process described in the figure is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Similar to... Figure 4B The CTU depicted in the image may include, for example, a CB for each chromaticity sample and two corresponding CBs for the luminance sample, as well as syntax elements for encoding the samples of the CB. In a monochrome image or an image with three separate color planes, a CU may include a single CB and a syntax structure for encoding the samples of the CB. It should be noted that... Figure 4C and Figure 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree partitioning / ternary tree partitioning / binary tree partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to a binary and / or ternary tree structure. Figure 4E As shown, a CB with width W and height H has five possible segmentation types: quadrilateral segmentation, horizontal binary segmentation, vertical binary segmentation, horizontal ternary segmentation, and vertical ternary segmentation.

[0086] In some implementations, the video encoder 20 may further segment the CB of the CU into one or more M×N PBs. A PB is a rectangular (square or non-square) block of samples to which the same prediction (inter-frame, intra-frame, etc.) is applied. The PU of the CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements for predicting the PBs. In a monochrome image or an image with three separate color planes, a PU may include a single PB and a syntax structure for predicting the PBs. The video encoder 20 may generate predicted luma blocks, predicted Cb blocks, and predicted Cr blocks for each PU of the CU, representing the luma PB, Cb PB, and Cr PB.

[0087] In addition, such as Figure 4C As shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. The TU of the CU can include a TB for luminance samples, two corresponding TBs for chrominance samples, and syntax elements for transforming the TBs. Therefore, each TU of the CU can be associated with a luminance TB, a Cb TB, and a Cr TB. In some examples, the luminance TB associated with a TU can be a sub-block of the CU's luminance residual block. The Cb TB can be a sub-block of the CU's Cb residual block. The Cr TB can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, the TU can include a single TB and syntax structures for transforming the samples of that TB.

[0088] Figure 5 This is a schematic diagram illustrating a video coding framework based on bitrate control and MCTF.

[0089] In a video coding framework, the bitrate control module typically guides the encoder on the encoding degree of each image block in the video image, thereby maximizing the video compression efficiency. The bitrate control module is usually located before video encoding and compression, such as... Figure 5As shown, the bitrate control module first analyzes the input source video to determine the quantization parameters used to control the image compression quality. The quantization parameters and image quality are inversely correlated.

[0090] The bitrate control module calculates the corresponding quantization parameter difference (deltaQp) for each frame of the video and each 16×16 block within each frame, based on the characteristics of the video content. deltaQp can be used to control the encoder's encoding level for the corresponding region. deltaQp is negatively correlated with image encoding quality; the smaller the deltaQp, the more bitrate is allocated to the current 16×16 block for encoding, and the better the quality of the image block needs to be encoded, and vice versa.

[0091] MCTF is a pre-encoding filtering algorithm widely used by many codec standards (such as HEVC, VVC, Audio and Video Codec Standard (AVS), AV1, AV2, etc.). This filtering algorithm is used to remove temporal noise from the video before encoding, thereby improving the overall encoding compression rate of the video.

[0092] like Figure 5 As shown, the MCTF filtering module is located after the bitrate control module. The MCTF filtering module performs MCTF on the source video to obtain the MCTF-processed video, that is, the source video has been filtered and modified. Then the encoding module encodes the MCTF-processed video.

[0093] In HEVC and VVC, the encoder first divides the video into multiple Groups of Pictures (GOPs), and then compresses the GOPs. Within each GOP, the images are compressed according to a fixed encoding order. Figure 6 The encoding structures of GOP4 and GOP8 are shown. GOP4 represents a group of images containing 5 images, and GOP8 represents a group of images containing 9 images. Figure 6 In this diagram, digital blocks represent images, and the numbers within a block represent the image's encoding order. The number on the x-axis corresponding to each digital block represents its Picture Order Count (POC), the number on the y-axis represents its encoding layer number within the Group of Pictures (GOP), and the image pointed to by the arrow of each digital block represents its reference image. For example, referencing... Figure 6 (a) According to the encoding order, the images in a GOP can be divided into multiple encoding layers. The first two frames encoded are in layer 0, the next frame encoded is in layer 1, and so on for subsequent frames. The larger the GOP, the more encoding layers it has.

[0094] Generally, MCTF takes effect at a lower coding level within a GOP. For example, Figure 6 In (b), only frames located at layers 0 and 1 in GOP8 will perform MCTF. This is because these frames are more important in the temporal reference structure.

[0095] MCTF can be mainly decomposed into two modules: Motion Compensation (MC) module and Temporal Filter (TF) module. Figure 7 A schematic diagram of MCTF is shown. Assume the current frame (denoted as...) If MCTF is required, the filtering reference frame for It must first be determined, which is generally four frames before and after it (denoted as ). , , , , , , , ), and in turn to Block-level motion estimation is performed with the eight adjacent filtered reference frames, such as... Figure 7 As shown, the purpose is to search for The motion vector (MV) is the position offset of the most similar block among the eight filtered reference frames for each 8×8 image patch. The MV is the position offset of the similar block (also called the matching block or reference block) relative to the current patch. For example, if there are eight filtered reference frames, then... Each 8x8 image block should have 8 MVs, such as Figure 7 In , , , , , , and To accelerate this process, MCTF can employ a hierarchical pyramid architecture to quickly derive the MV for each 8×8 image block. It's important to note that in practical applications, the number of filter reference frames can be flexibly adjusted based on the specific application and is not necessarily limited to 8.

[0096] In the time-domain filtering of MCTF, Each 8×8 image block is weighted and summed with 8 reference blocks obtained from the 8 adjacent filtered reference frames (i.e., a total of 9 image blocks) according to a certain ratio (each image block has a corresponding weight w (such as...). Figure 7 In , , , , , , and (w) is an 8×8 weight block, corresponding to the size of the image patch. In the weighted summation, each pixel in the image patch is multiplied by the weight value at the corresponding position in the weight block. The result of the MCTF filtering is a filtered 8×8 image patch, which will replace... The original image patch in the image.

[0097] Figure 8 A schematic diagram of the bit rate control block and the MCTF filter block is shown.

[0098] In the encoding / decoding standard, a complete frame of image is first divided into multiple CTUs according to the size of the Code Tree Unit (CTU) for encoding. The MCTF filtering module filters the image according to 8×8 blocks (which can be called MCTF filtering blocks or filtering blocks). The rate control module derives a deltaQp for each 16×16 block of the image (i.e., a unit block of the rate control module, which can be called a rate control unit block). Therefore, one rate control unit block can correspond to four 8×8 MCTF filtering blocks, such as... Figure 8 As shown. The MCTF filtering module also filters the image in units of CTUs. For example, assuming the size of a CTU is 128×128, each CTU can contain 8×8=64 bitrate control blocks and 16×16=256 MCTF filtering blocks. The above example is merely exemplary, and this disclosure is not limited thereto.

[0099] As mentioned above, the bitrate control module is located before the MCTF filtering module. The source video after MCTF has already been filtered and modified. In this case, the quantization parameters derived from the source video to control the compression quality of the source video are used in the video image after MCTF. This results in a certain error in the quantization parameters, which in turn has a significant impact on the compression efficiency of the encoder.

[0100] Therefore, to address the aforementioned problems, this disclosure proposes a correction method for adjusting the MCTF block-level filter weights using quantization parameter information derived from the bitrate control module. This optimizes the QP error caused by the inconsistency between the video used for bitrate control and the actual encoded video due to the introduction of MCTF, thereby improving video encoding and decoding performance. In this disclosure, "image" and "frame" are used interchangeably.

[0101] Figure 9 This is a flowchart illustrating an image encoding method according to some embodiments of the present disclosure.

[0102] Reference Figure 9In step 901, the first quantization parameter information of the bitrate control block in the image to be filtered is obtained, wherein the bitrate control block represents a processing unit used for bitrate allocation and image quality control. Figure 9 The method shown is described using the currently processed bitrate control block as an example. The method is executed for each bitrate control block in a frame of image.

[0103] According to embodiments, the image to be filtered can be an image located in a lower coding layer within a Group of Pictures (GOP). For example, the image to be filtered can be a frame located in layers 0 and 1 of GOP8. The image to be filtered can be an image to which MCTF will be performed. The above examples are merely exemplary, and this disclosure is not limited thereto.

[0104] According to an embodiment, the bitrate control unit block can be a unit block used for bitrate control. The bitrate control unit block can be a block for which the bitrate control module calculates the corresponding quantization parameter information. For example, the bitrate control unit block can be... Figure 8 The example shown is a 16×16 block. The above example is merely exemplary, and this disclosure is not limited thereto.

[0105] According to an embodiment, the first quantization parameter information may be the quantization parameter difference (deltaQp) calculated by the rate control module for the rate control block. deltaQp can be used to control the encoding degree of the encoder for the corresponding rate control block.

[0106] In step 902, the filter weights of the filter block to be subjected to motion-compensated temporal filtering in the bit rate control block are adjusted based on the first quantization parameter information. In this disclosure, the filter block to be subjected to motion-compensated temporal filtering may be referred to as an MCTF filter block.

[0107] According to an embodiment, a bit rate control unit block may include at least one MCTF filter block. For example, Figure 8 The relationship between the rate control unit and the MCTF filter block is shown in the diagram, but this disclosure is not limited thereto. Furthermore, the sizes of the rate control unit and the MCTF filter block can be set differently.

[0108] According to an embodiment, a weight adjustment factor for adjusting the filter weights of the MCTF filter block can be determined based on the first quantization parameter information of the bit rate control block, and then the filter weights can be adjusted based on the weight adjustment factor.

[0109] As an example, the value of the weight adjustment factor can be determined based on the value of the first quantization parameter information of the rate control block. The above example is merely illustrative, and this disclosure is not limited thereto. When the rate control block includes multiple MCTF filter blocks, the multiple MCTF filter blocks can share the weight adjustment factor determined based on the first quantization parameter information. That is, the determined weight adjustment factor can be used to adjust the filtering weights of each MCTF filter block in the rate control block.

[0110] According to another embodiment, second quantization parameter information of a coding tree unit including a rate control unit can be obtained, and a weight adjustment factor is determined based on the first and second quantization parameter information. Here, the coding tree unit including the currently processed rate control unit can be understood as the currently processed CTU.

[0111] As an example, a coding tree unit may include at least one code rate control unit block. For example, Figure 8 The diagram illustrates the relationship between the CTU and the rate control block. The rate control module can determine the corresponding deltaQp for each rate control block in a frame. In this case, the deltaQp of each rate control block in the CTU can be obtained. Then, the deltaQp of each rate control block can be summed and averaged to obtain the second quantization parameter information of the coding tree unit. Here, the second quantization parameter information can also be referred to as the average quantization parameter information or the average deltaQp.

[0112] For example, the second quantization parameter information can be calculated according to the following equation (1). .

[0113] (1) in, This indicates the number of rate control unit blocks in the CTU. This represents the deltaQp of the i-th rate control unit block. The above example is merely exemplary, and this disclosure is not limited thereto. For example, after obtaining the deltaQp of each rate control unit block in the CTU, different calculations (such as weighted averaging) can be performed on these deltaQp to obtain the second quantization parameter information.

[0114] After obtaining the second quantization parameter information, the weight adjustment factor can be determined based on the relationship between the first quantization parameter information and the second quantization parameter information.

[0115] As an example, a ratio between a first quantization parameter and a second quantization parameter can be defined; this ratio is compared with a first threshold and a second threshold to determine a weight adjustment factor, wherein the first threshold is less than or equal to the second threshold. If the ratio is less than the first threshold, the weight adjustment factor is set to a preset value; if the ratio is greater than the second threshold, the weight adjustment factor is set to the second threshold; if the ratio is greater than or equal to the first threshold and less than or equal to the second threshold, the weight adjustment factor is set to a value equal to the ratio.

[0116] For example, the (current) bitrate control block can be calculated according to the following equation (2). and the CTU including the rate control unit block The ratio between .

[0117] (2) Next, the bitrate control block can be configured. Amplitude limiting is applied to determine the weight adjustment factor used for MCTF. For example, the following equation (3) can be used to determine the weight adjustment factor.

[0118] (3) in, Represents the amplitude limiting function. Indicates the first threshold. This indicates the second threshold.

[0119] like Less than ,but It can be set to 0.5; if Greater than ,but Can be set to equal to ;otherwise Can be set to equal to . and It can be a real number, and simultaneously satisfies .

[0120] According to an embodiment, a first threshold and a second threshold can be determined based on the frame type of the image to be filtered. For example, in the case of an I-frame, It can be set to 0.2. It can be set to 3.0. In the case of P-frames, It can be set to 0.3. It can be set to 2.0. In the case of B-frames, It can be set to 0.4. It can be set to 1.5. The example values ​​above are exemplary, and this disclosure is not limited thereto.

[0121] For I-frames, P-frames, and B-frames, their corresponding and The following size relationship can be satisfied:

[0122]

[0123] The above examples are merely illustrative and can be adapted to suit specific circumstances; this disclosure is not limited thereto.

[0124] After obtaining the weight adjustment factor for the bitrate control block, the filtering weights of all MCTF filtering blocks in the bitrate control block can be adjusted based on the weight adjustment factor. For example, the following equation (3) can be used to adjust the filtering weights.

[0125] (3) in, This represents the adjusted filter weights of the k-th MCTF filter block. This represents the filter weight of the k-th MCTF filter block before adjustment.

[0126] In step 903, motion-compensated temporal filtering is performed on the filter block based on the adjusted filter weights.

[0127] According to an embodiment, the operations included in step 902 above can be performed on each CTU and each bit rate control block in the image to be filtered.

[0128] According to Figure 7 The method shown executes MCTF. It can... Figure 7 The filter weights of the MCTF filter block in the current image to be filtered are replaced with the adjusted filter weights mentioned above, and the filter weights of the reference block do not need to be adjusted.

[0129] In step 904, encoding is performed based on the filtering result.

[0130] As an example, for each rate control unit block in the image to be filtered, the MCTF-encoded blocks in that rate control unit block can be encoded according to the first quantization parameter information of the rate control unit block.

[0131] According to another embodiment of this disclosure, before performing step 902, it is first determined whether the filter weights need to be adjusted.

[0132] As an example, the second quantization parameter information of each coding tree unit in the image to be filtered can be obtained. The number of coding tree units in the image to be filtered whose second quantization parameter information is greater than the second quantization parameter information of the coding tree units including the (current) bitrate control unit block can be determined. The ratio between this number and the total number of coding tree units in the image to be filtered can be determined. If this ratio is greater than a preset threshold, the filtering weights of the filtering block are adjusted. If this ratio is less than or equal to the preset threshold, the adjustment of the filtering weights of the filtering block is skipped.

[0133] As an example, firstly, for all CTUs in a frame, the statistical results for each CTU are obtained based on equation (1). Regarding the current CTU Can be recorded as .

[0134] Then, for the current CTU, calculate the difference between the current CTU and the current CTU. Larger number of CTUs The number of all CTUs in this frame proportion For example, equations (4) and (5) below can be used to calculate... .

[0135] (4) (5) Equation (4) represents the current CTU's With other CTUs in the frame In comparison, at other CTUs Greater than the current CTU If the condition is met, increment the count by 1; otherwise, do not count.

[0136] according to The value is compared with the preset threshold skipThre. At that time, it can be determined that the filtering weights of the MCTF filtering block of the current CTU will be adjusted.

[0137] For a custom threshold, the value range is [0,1], for example, The above examples are merely illustrative, and this disclosure is not limited thereto.

[0138] Figure 10 This is a flowchart illustrating another image encoding method according to some embodiments of the present disclosure.

[0139] Reference Figure 10After determining that MCTF will be performed on the current image, in step 1001, the second quantization parameter information of each CTU in the current image, such as the average deltaQp of each CTU, can be calculated.

[0140] The MCTF filtering module can filter the current image in units of CTU.

[0141] In step 1002, for the current CTU, the ratio between the number of CTUs in the current image that have a larger average deltaQp than the current CTU and the total number of CTUs is calculated.

[0142] In step 1003, it is determined whether the determined ratio is greater than or equal to a preset threshold.

[0143] If the determined proportion is greater than or equal to a preset threshold, proceed to step 1004 to adjust the filter weights of each MCTF filter block within the current CTU. The operation of adjusting the filter weights can be referred to the description of step 902 above, and will not be repeated here.

[0144] If the determined proportion is less than the preset threshold, proceed to step 1005, and do not adjust the filtering weight of each MCTF filter block in the current CTU.

[0145] In step 1006, MCTF is performed on the current CTU and the filtering result is encoded.

[0146] According to embodiments of this disclosure, a method for adjusting the filtering weights used in MCTF is proposed based on the ratio of the average deltaQp at the CTU level to the deltaQp at the rate control block level. Furthermore, a method for skipping the filtering weight adjustment of this disclosure is proposed based on the relationship between the average deltaQp across CTU levels. This disclosure solves the problem of DeltaQp errors caused by the MCTF filtering module being located after the rate control module, thereby improving compression efficiency.

[0147] Figure 11 This is a block diagram illustrating an image encoding apparatus according to some embodiments of the present disclosure.

[0148] Reference Figure 11 The image encoding device 1100 may include an acquisition module 1101, a filtering module 1102, and an encoding module 1103. Optionally, the image encoding device 1100 may also include a determination module 1104 (not shown). Figure 11 The number and names of the models described are merely illustrative, and these modules can be split and combined according to the actual situation.

[0149] The acquisition module 1101 can acquire the first quantization parameter information of the bitrate control block in the image to be filtered, wherein the bitrate control block represents a processing unit used for bitrate allocation and image quality control.

[0150] The filtering module 1102 can adjust the filtering weights of the filtering block to be subjected to motion compensation temporal filtering in the bit rate control block based on the first quantization parameter information, and perform motion compensation temporal filtering on the filtering block based on the adjusted filtering weights.

[0151] The encoding module 1103 can encode the filter block that has undergone motion-compensated time-domain filtering based on the first quantization parameter information.

[0152] According to an embodiment, the filtering module 1102 can determine a weight adjustment factor for adjusting the filtering weights based on the first quantization parameter information; and adjust the filtering weights of the filtering block based on the weight adjustment factor.

[0153] According to an embodiment, the filtering module 1102 can obtain second quantization parameter information of the coding tree unit including the bit rate control unit block; and determine the weight adjustment factor based on the first quantization parameter information and the second quantization parameter information.

[0154] According to the embodiment, the filtering module 1102 can obtain the first quantization parameter information of each bit rate control block included in the coding tree unit; and determine the second quantization parameter information of the coding tree unit based on the first quantization parameter information of each bit rate control block.

[0155] According to an embodiment, the filtering module 1102 can first sum and then average the first quantization parameter information of each bit rate control unit block to obtain the second quantization parameter information of the coding tree unit.

[0156] According to an embodiment, the filtering module 1102 can determine the ratio between the first quantization parameter information and the second quantization parameter information; compare the ratio with a first threshold and a second threshold to determine the weight adjustment factor, wherein the first threshold is less than or equal to the second threshold.

[0157] According to an embodiment, the filtering module 1102 may set the weight adjustment factor to a preset value when the ratio is less than the first threshold; set the weight adjustment factor to the second threshold when the ratio is greater than the second threshold; and set the weight adjustment factor to a value equal to the ratio when the ratio is greater than or equal to the first threshold and less than or equal to the second threshold.

[0158] According to an embodiment, the first threshold and the second threshold are determined based on the frame type of the image to be filtered.

[0159] According to an embodiment, the determining module 1104 can obtain the second quantization parameter information of each coding tree unit in the image to be filtered; determine the number of coding tree units in the image to be filtered whose second quantization parameter information is greater than the second quantization parameter information of the coding tree units including the bit rate control unit block; determine the ratio between the number and the total number of coding tree units in the image to be filtered; and if the ratio is greater than a preset threshold, determine to adjust the filtering weight of the filtering block.

[0160] According to an embodiment, the determining module 1104 may determine to skip adjusting the filtering weight of the filter block if the ratio is less than or equal to the preset threshold.

[0161] For information on the operations performed by the image encoding device, please refer to the above. Figure 9 and Figure 10 The description will not be repeated here.

[0162] Figure 12 A computing environment 1610 coupled to a user interface 1650 is shown. The computing environment 1610 may be part of a data processing server. The computing environment 1610 includes a processor 1620, memory 1630, and input / output (I / O) interface 1640.

[0163] Processor 1620 typically controls the overall operation of computing environment 1610, such as operations associated with display, data acquisition, data communication, and image processing. Processor 1620 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Furthermore, processor 1620 may include one or more modules that facilitate interaction between processor 1620 and other components. The processor may be a central processing unit (CPU), microprocessor, microcontroller, graphics processing unit (GPU), etc.

[0164] Memory 1630 is configured to store various types of data to support the operation of computing environment 1610. Memory 1630 may include predefined software 1632. Examples of such data include instructions for any application or method operating on computing environment 1610, video datasets, image data, etc. Memory 1630 can be implemented using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0165] I / O interface 1640 provides an interface between processor 1620 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). I / O interface 1640 can be coupled with encoders and decoders.

[0166] In embodiments, a non-transitory computer-readable storage medium or computer program product is also provided, including, for example, a plurality of programs in memory 1630 and / or a storage bitstream, said plurality of programs being executable by processor 1620 to perform the above-described encoding or decoding methods, said bitstream being generated by the above-described encoding method and / or to be decoded by the above-described decoding method. For example, the computer program product may include a non-transitory computer-readable storage medium. In one example, said plurality of programs may be executable by processor 1620 to (e.g., from...) Figure 2 The video encoder 20 in the process receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1620 to perform the above-described decoding method based on the received bitstream or data stream. In another example, the plurality of programs can be executed by the processor 1620 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 1620 to (e.g., to...) Figure 3 The video decoder 30 in the middle transmits or stores the bit stream or data stream. Alternatively, the bit stream or data stream may be stored in a non-transitory computer-readable storage medium or a computer program product.

[0167] In an embodiment, a bitstream (e.g., including encoded video information) generated by the above encoding method and / or decoded by the above decoding method is provided.

[0168] In an embodiment, a computing device is also provided, comprising: one or more processors (e.g., processor 1620); and a non-transitory computer-readable storage medium or memory 1630 therein storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs. In one example, the one or more processors are configured to perform the encoding method described above to generate a bitstream when executing the plurality of programs, and the computing device may further include a transmitter configured to transmit the bitstream. In an alternative example, the one or more processors are configured to perform the encoding method described above to generate a bitstream when executing the plurality of programs, and to transmit or store the bitstream. In one example, the bitstream will be decoded using the decoding method described above.

[0169] In an embodiment, the computing environment 1610 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0170] In one embodiment, a method for storing a bitstream is also provided, comprising: storing the bitstream on a non-transitory computer-readable storage medium, wherein the bitstream is generated by the above-described encoding method and / or decoded by the above-described decoding method. In another embodiment, a method for storing a bitstream or a method for encoding video data is also provided, comprising: performing the above-described encoding method to generate a bitstream, and storing the bitstream on a non-transitory computer-readable storage medium. In one example, the bitstream will be decoded by the above-described decoding method.

[0171] In one embodiment, a method for transmitting a bitstream is also provided, the bitstream being generated by the above-described encoding method and / or decoded by the above-described decoding method. In another embodiment, a method for transmitting a bitstream or for encoding video data is also provided, comprising: performing the above-described encoding method to generate a bitstream, and transmitting the bitstream to a decoder. In one example, the bitstream will be decoded by the above-described decoding method. In another embodiment, a method for receiving a bitstream is also provided, the bitstream being generated by the above-described encoding method and / or decoded by the above-described decoding method.

[0172] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to this disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0173] Unless otherwise specifically stated, the order of steps in the method according to this disclosure is intended to be illustrative only, and the steps of the method according to this disclosure are not limited to the specific order described above, but may be changed according to actual circumstances. Furthermore, at least one step in the method according to this disclosure may be adjusted, combined, or omitted as needed.

[0174] The examples chosen and described are intended to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure, and preferably to utilize the basic principles and various embodiments with various modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. An image encoding method, comprising: Obtain the first quantization parameter information of the bitrate control block in the image to be filtered, wherein the bitrate control block represents a processing unit used for bitrate allocation and image quality control; The filtering weights of the filtering blocks in the bitrate control block to which motion compensation temporal filtering will be performed are adjusted based on the first quantization parameter information. Motion-compensated temporal filtering is performed on the filter block based on the adjusted filter weights; and Encoding is performed based on the filtering results.

2. The method according to claim 1, wherein, Based on the first quantization parameter information, the filter weights of the filter blocks in the bit rate control block to which motion compensation temporal filtering will be performed are adjusted, including: A weight adjustment factor for adjusting the filter weights is determined based on the first quantization parameter information; The filtering weights of the filter block are adjusted based on the weight adjustment factor.

3. The method according to claim 2, wherein, Determining the weight adjustment factor for the filter block based on the first quantization parameter information includes: Obtain the second quantization parameter information of the coding tree unit including the bit rate control unit block; The weight adjustment factor is determined based on the first quantization parameter information and the second quantization parameter information.

4. The method according to claim 3, wherein, Obtaining the second quantization parameter information of the coding tree unit including the bit rate control block includes: Obtain the first quantization parameter information of each code rate control unit block included in the coding tree unit; The second quantization parameter information of the coding tree unit is determined based on the first quantization parameter information of each bit rate control unit block.

5. The method according to claim 4, wherein, The second quantization parameter information of the coding tree unit is determined based on the first quantization parameter information of each bit rate control block, including: The first quantization parameter information of each bit rate control unit block is first summed and then averaged to obtain the second quantization parameter information of the coding tree unit.

6. The method according to claim 3, wherein, Determining the weight adjustment factor based on the first quantization parameter information and the second quantization parameter information includes: Determine the ratio between the first quantization parameter information and the second quantization parameter information; The ratio is compared with a first threshold and a second threshold to determine the weight adjustment factor, wherein the first threshold is less than or equal to the second threshold.

7. The method according to claim 6, wherein, Comparing the ratio with a first threshold and a second threshold to determine the weight adjustment factor includes: If the ratio is less than the first threshold, the weight adjustment factor is set to a preset value. If the ratio is greater than the second threshold, the weight adjustment factor is set to be equal to the second threshold; If the ratio is greater than or equal to the first threshold and less than or equal to the second threshold, the weight adjustment factor is set to a value equal to the ratio.

8. The method according to claim 6, wherein, The first threshold and the second threshold are determined based on the frame type of the image to be filtered.

9. The method according to claim 1, wherein, Based on the first quantization parameter information, the filter weights of the filter blocks in the bit rate control block to which motion compensation temporal filtering will be performed are adjusted, including: Obtain the second quantization parameter information of each coding tree unit in the image to be filtered; The number of coding tree units whose second quantization parameter information in the image to be filtered is greater than the second quantization parameter information of the coding tree units including the bit rate control unit block is determined. Determine the ratio between the quantity and the total number of coding tree units in the image to be filtered; If the ratio is greater than a preset threshold, the filtering weights of the filter block are adjusted.

10. The method of claim 9, further comprising: If the ratio is less than or equal to the preset threshold, the adjustment of the filtering weight of the filter block is skipped.

11. An image encoding apparatus, comprising: The acquisition module is configured to: acquire the first quantization parameter information of the bitrate control block in the image to be filtered, wherein the bitrate control block represents a processing unit used for bitrate allocation and image quality control; The filtering module is configured to: adjust the filtering weights of the filtering blocks in the bitrate control block to be subjected to motion-compensated temporal filtering based on the first quantization parameter information, and perform motion-compensated temporal filtering on the filtering blocks based on the adjusted filtering weights; and The encoding module is configured to encode the filter block that has undergone motion-compensated time-domain filtering based on the first quantization parameter information.

12. An electronic device, comprising: At least one processor; At least one memory that stores computer-executable instructions. The computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the image encoding method as described in any one of claims 1 to 10.

13. A computer-readable storage medium storing a bit stream and instructions, wherein, When executed by a computing device having one or more processors, the instructions cause the one or more processors to perform the image encoding method according to any one of claims 1 to 10 to generate the bitstream.

14. A computer program product, wherein, When executed by a computing device having one or more processors, the instructions in the computer program product cause the one or more processors to perform the image encoding method according to any one of claims 1 to 10 to generate a bitstream.

15. A method for storing a bit stream, comprising: Perform the image encoding method according to any one of claims 1 to 10 to generate a bitstream; as well as Store the bit stream.

Images