Video encoding and decoding method and device

Abstract

The invention provides a video encoding and decoding method and a device thereof. A video encoding method may include: obtaining a reconstructed image for a current frame; based on a first-level block of the reconstructed image, using a neural network to obtain a deblocking filtering parameter for the first-level block; and performing deblocking filtering on the first level block using the deblocking filtering parameter.

Description

Technical Field

[0001] The present disclosure relates to video coding and compression, and more particularly to a video coding method and apparatus thereof, an electronic device and a non-transitory computer-readable storage medium. Background Art

[0002] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send and receive or otherwise transmit digital video data through a communication network, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, before the video data is transmitted or stored, the video data can be compressed using a video codec according to one or more video codec standards. For example, video codec standards include general video codec (VVC), joint exploration test model (JEM), high efficiency video codec (HEVC / H.265), advanced video codec (AVC / H.264), moving picture expert group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the inherent redundancy in video data. Video codecs are intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality. Summary of the invention

[0003] Examples of the present disclosure provide a video coding method and apparatus, an electronic device, and a non-transitory computer-readable storage medium for loop filtering in video coding.

[0004] According to a first aspect of the present disclosure, a video encoding method is provided, comprising: obtaining a reconstructed image for a current frame; based on a first-level block of the reconstructed image, obtaining deblocking filtering parameters for the first-level block using a neural network; and performing deblocking filtering on the first-level block using the deblocking filtering parameters.

[0005] Optionally, based on the first-level block of the reconstructed image, a neural network is used to obtain deblocking filtering parameters for the first-level block, including: based on the encoding features of the first-level block and at least one of the reconstructed blocks obtained by expanding the first-level block, using the neural network to obtain the deblocking filtering parameters.

[0006] Optionally, the deblocking filter parameters include at least one of a filter switch decision threshold, a pixel difference threshold, a weak filtering decision value and a strong filtering decision value, wherein the weak filtering decision value is used to further adjust the weak filtering process in the deblocking filter, and the strong filtering decision value is used to further adjust the strong filtering process in the deblocking filter.

[0007] Optionally, the first-level block is a slice, a tile, a coding tree unit or a coding unit.

[0008] Optionally, the method also includes: calculating first distortion information between a first filtering result obtained after performing deblocking filtering on the first-level block using the deblocking filtering parameters and original pixels of the first-level block; calculating second distortion information between a second filtering result obtained after performing deblocking filtering on the first-level block using default filtering parameters and original pixels of the first-level block; and determining, based on the first distortion information and the second distortion information of the first-level block, a first syntax element for indicating whether to perform deblocking filtering on the first-level block using the deblocking filtering parameters.

[0009] Optionally, the method also includes: determining the first distortion information and the second distortion information for each block in the second-level block including the first-level block; based on the first distortion information and the second distortion information of each block, determining a second syntax element for indicating whether to enable the use of the neural network to predict the deblocking filter parameters for the second-level block and / or a third syntax element for indicating whether to enable the selection of the deblocking filter parameters for the blocks in the second-level block.

[0010] Optionally, the method further includes: when the first distortion information of each block in the second-level blocks including the first-level blocks is all greater than the second distortion information or the first distortion information is all less than the second distortion information, not encoding the first syntax element of each block and not sending the first syntax element of each block by signal; when there is at least one block in the second-level blocks whose first distortion information is equal to the second distortion information, encoding the first syntax element of each block and sending the first syntax element of each block by signal.

[0011] Optionally, the method further includes: when there is at least one block in the second-level blocks whose first distortion information is equal to the second distortion information, determining a value of the first syntax element of the at least one block based on a codeword length of the first syntax element for each block.

[0012] Optionally, the method further includes: when the first distortion information of each block in the second-level block including the first-level block is all greater than the second distortion information, determining that the value of the second syntax element is 1 and the value of the third syntax element is 0; when the first distortion information of each block is all less than the second distortion information, determining that the value of the second syntax element is 0 and the value of the third syntax element is 0; when there is at least one block in the second-level block whose first distortion information is equal to the second distortion information, determining that the value of the second syntax element is 1 and the value of the third syntax element is 1.

[0013] Optionally, deblocking filtering is performed on the first-level block using the deblocking filtering parameters, including: adjusting the brightness short-tap weak filtering process based on the weak filtering decision value, wherein, when the weak filtering decision value is a first value, two rows of pixels on both sides of the boundary of the first-level block are corrected; when the weak filtering decision value is a second value, one row of pixels on both sides of the boundary of the first-level block are corrected; when the weak filtering decision value is a third value, the two rows of pixels on both sides of the boundary of the first-level block are not corrected.

[0014] According to a second aspect of the present disclosure, a video decoding method is provided, comprising: receiving a bit stream and obtaining a reconstructed image for a current frame based on the bit stream; based on a first-level block of the reconstructed image, obtaining a deblocking filtering parameter for the first-level block using a neural network; and performing deblocking filtering on the first-level block using the deblocking filtering parameter.

[0015] Optionally, based on the first-level block of the reconstructed image, a neural network is used to obtain deblocking filtering parameters for the first-level block, including: based on the encoding features of the first-level block and at least one of the reconstructed blocks obtained by expanding the first-level block, using the neural network to obtain the deblocking filtering parameters.

[0016] Optionally, the deblocking filter parameters include at least one of a filter switch decision threshold, a pixel difference threshold, a weak filtering decision value and a strong filtering decision value, wherein the weak filtering decision value is used to further adjust the weak filtering process in the deblocking filter, and the strong filtering decision value is used to further adjust the strong filtering process in the deblocking filter.

[0017] Optionally, the first-level block is a slice, a tile, a coding tree unit or a coding unit.

[0018] Optionally, based on the first-level block of the reconstructed image, a deblocking filtering parameter for the first-level block is obtained using a neural network, including: obtaining from the bitstream a first syntax element for indicating whether to use the deblocking filtering parameter to perform deblocking filtering on the first-level block; when a value of the first syntax element is 1, based on the first-level block, the deblocking filtering parameter is obtained using the neural network; when the value of the first syntax element is 0, the deblocking filtering parameter is not obtained using the neural network.

[0019] Optionally, the method also includes: obtaining from the bitstream a second syntax element for indicating whether to use the neural network to predict the deblocking filter parameters for a second-level block including the first-level block and / or a third syntax element for indicating whether to select the deblocking filter parameters for the second-level block.

[0020] Optionally, the method also includes: when the second syntax element is 0 and the value of the third syntax element is 0 or the third syntax element is not obtained, deriving the value of the first syntax element of each block in the second-level block as 0; when the second syntax element is 1 and the value of the third syntax element is 0 or the third syntax element is not obtained, deriving the value of the first syntax element of each block as 1; when the values ​​of the second syntax element and the third syntax element are both 1, obtaining the first syntax element of each block from the bitstream.

[0021] Optionally, deblocking filtering is performed on the first-level block using the deblocking filtering parameters, including: performing the luminance short-tap weak filtering process based on the weak filtering decision value, wherein, when the weak filtering decision value is a first value, two rows of pixels on both sides of the boundary of the first-level block are corrected; when the weak filtering decision value is a second value, one row of pixels on both sides of the boundary of the first-level block are corrected; when the weak filtering decision value is a third value, the two rows of pixels on both sides of the boundary of the first-level block are not corrected.

[0022] According to a third aspect of the present disclosure, a device for video decoding is provided, comprising: one or more processors; and a memory, coupled to the one or more processors and configured to store instructions that can be executed by the one or more processors, wherein the one or more processors are configured to perform the video decoding method according to the present disclosure when executing the instructions.

[0023] According to a fourth aspect of the present disclosure, a device for video encoding is provided, comprising: one or more processors; and a memory, coupled to the one or more processors and configured to store instructions that can be executed by the one or more processors, wherein the one or more processors are configured to execute the video encoding method according to the present disclosure when executing the instructions.

[0024] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium for storing computer-executable instructions is provided, wherein the computer-executable instructions, when executed by one or more computer processors, cause the one or more computer processors to receive a bit stream and perform a video decoding method according to the present disclosure based on the bit stream.

[0025] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium for storing computer-executable instructions is provided. When the computer-executable instructions are executed by one or more computer processors, the one or more computer processors execute the video encoding method according to the present disclosure to encode first to third syntax elements into a bit stream and send the bit stream.

[0026] According to a seventh aspect of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream includes: video data to be decoded by the video decoding method according to the present disclosure; and / or video data generated by the video encoding method according to the present disclosure.

[0027] According to an eighth aspect of the present disclosure, a method for storing a bitstream is provided, comprising: storing the bitstream on a computer-readable storage medium, wherein the bitstream includes video data decoded according to the video decoding method of the present disclosure, and / or video data generated by the video encoding method of the present disclosure.

[0028] It is to be understood that both the foregoing general description and the following detailed description are examples only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate examples according to the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0030] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0031] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0032] Figure 3is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0033] Figure 4 is a flowchart of a deblocking filtering process according to some embodiments of the present disclosure.

[0034] Figure 5 is an overall flow chart of deblocking filtering according to some embodiments of the present disclosure.

[0035] Figure 6 is a flowchart of a video encoding method according to some embodiments of the present disclosure.

[0036] Figure 7 is a flowchart of a video decoding method according to some embodiments of the present disclosure.

[0037] Figure 8 is a diagram illustrating a computing environment coupled with a user interface according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0038] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the drawings are used to distinguish objects, but not to describe any specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0040] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0041] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0042] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0043] like Figure 1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0044] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0045] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0046] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0047] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0048] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present application. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0049] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a partition unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0050] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0051] like Figure 2 As shown in , after receiving the video data, the partitioning unit 45 within the prediction processing unit 41 partitions the video data into video blocks. This partitioning may also include partitioning the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0052] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0053] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0054] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0055] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0056] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0057] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0058] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0059] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0060] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0061] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0062] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0063] Furthermore, in some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0064] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. Video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (e.g., a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk).

[0065] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0066] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0067] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0068] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0069] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded.

[0070] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0071] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0072] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0073] For the current mainstream video coding and decoding standards (such as VVC standards, etc.), due to the use of block-based hybrid coding and decoding framework, the encoding or decoding process is as follows Figure 2 and Figure 3 As shown in the figure, after the compressed image is reconstructed, there will be problems of image quality degradation caused by artificial processing such as block effects and ringing effects. In order to improve the image quality, VVC uses loop filtering technology. The loop filter can be placed before the decoded picture buffer. In the "prediction-reconstruction-filtering-prediction" loop, it can not only improve the quality of the current frame itself, but also improve the quality of subsequent references. The same processing flow can be used on the encoding side and the decoding side to achieve codec matching. Loop filtering can include deblocking filter (Deblocking Filter, DF), adaptive sample offset (Sample Adaptive Offset, SAO), adaptive loop filtering (Adaptive Loop Filtering, ALF), etc. Loop filtering can perform pixel domain processing on the reconstructed image.

[0074] The reconstructed image can be first subjected to deblocking filtering. Deblocking filtering mainly removes the discontinuity of the coding block (CU) boundary, transform block (TU) boundary, and prediction block (PU) boundary in the image. The deblocking filtering method in the HEVC and VVC standards has a certain degree of adaptability, and can make filtering decisions based on the video content and coding parameters, and no information needs to be encoded and transmitted during the filtering process. For example, the boundaries generated by intra-frame prediction coding of smooth content are strongly filtered, and the boundaries generated by coding of rich texture content are weakly filtered or not filtered. In addition, the deblocking filtering technology in HEVC and VVC only corrects the pixel values ​​of N rows or N columns on both sides of the coding block boundary, which enables it to support parallel processing of 8×8 blocks, and all vertical boundaries of the image are processed before the horizontal boundaries, and the decoding order of the coding blocks is more flexible.

[0075] Figure 4 is a flowchart of a deblocking filtering process according to some embodiments of the present disclosure.

[0076] Reference Figure 4 For the reconstructed image, the boundary information can be obtained according to the block division information. Here, the block can refer to CTU, PU, ​​TU, etc.

[0077] The maximum filter length may be determined according to a CTU boundary, a PU boundary, or a TU boundary. For example, the maximum filter length may be used to determine the number of taps of the filter to be used.

[0078] The boundary strength (BS) can be obtained according to the prediction mode, motion vector or reference frame of the block, etc. BS can also be called boundary filtering strength, which can be used to determine whether the block boundary needs to be filtered.

[0079] The pixel difference threshold (Tc or t c ) and other parameters to select the strength of the filter. The boundary pixel characteristics may include, for example, the content characteristics of the block boundary determined by the degree of change of the pixel values ​​in the boundary block (such as the texture of the block boundary), the degree of change of the pixels on both sides of the boundary, etc. For example, the texture and beta of the block boundary can be used to determine whether to turn on the filter switch. For another example, the filter strength can be determined based on the pixel values, beta and Tc on both sides of the boundary. The filter strength may refer to what intensity of filtering is required when it is determined that filtering is required. The above examples are only exemplary, and the present disclosure is not limited thereto.

[0080] After the filtering decision is made through the above operation, the filtering operation can be performed using parameters such as boundary strength, beta and Tc. The filtering operation can include a luminance long filtering operation, a luminance short tap strong filtering, a luminance short tap weak filtering, a chrominance strong filtering and a chrominance weak filtering.

[0081] For the brightness short tap weak filtering process, the pixel range and amplitude to be corrected are small, and the pixels that need to be corrected at most are 2 pixels on both sides of the boundary. Taking the vertical boundary as an example, it is necessary to perform filtering operations according to the specific conditions of each row of pixels at the vertical boundary. For example, for one of the rows, first determine whether to correct the first pixel p0 on the left side of the vertical boundary in the row and the first pixel q0 on the right side of the vertical boundary in the row, and then determine whether to correct the second pixel p1 on the left side of the vertical boundary in the row and the second pixel q1 on the right side of the vertical boundary in the row. In other words, the brightness short tap weak filtering process mainly includes two steps, the first step is to correct p0 and q0, and the second step is to correct p1 and q1. Filtering is performed in the above manner for each row of pixels at the vertical boundary.

[0082] A reconstructed image with removed blocking effects or ringing effects is obtained by performing deblocking filtering on the reconstructed image.

[0083] In the VVC standard, deblocking filtering supports parameter settings at the picture parameter set PPS level and slice level. For example, the deblocking filter parameters of each slice can be different, while the deblocking filter parameters of each CTU in a slice are the same. Deblocking filter parameters include, for example, a pixel difference threshold Tc (for example, ranging from -K to K, K is an integer), a filter switch decision threshold beta (for example, ranging from -K to K, K is an integer), where the value of K is 6 in the HEVC standard and 12 in the VVC standard. VTM and HM are the codec reference software for the VVC and HEVC standards, respectively. In HM and VTM, after the slice completes the reconstruction process, the candidates for beta and Tc are cycled to select the best slice-level beta and Tc.

[0084] In the current coding and decoding standards, in order to comply with the bitstream standards, the deblocking filtering process is fixed and the filtering effect is limited, especially the fixed weak filtering process has a poor effect.

[0085] In addition, the deblocking filter uses slice-level parameter selection and processing, resulting in poor encoding and decoding performance. In the case of slice-level, the slice needs to complete the reconstruction process as a whole before parameter selection can be performed. This is not applicable in actual commercial codecs. Because in order to achieve the maximum computing speed and compression efficiency, commercial codecs will use a multi-threaded approach for encoding and decoding, that is, frames with a reference relationship can be encoded and decoded at the same time. If the slice-level parameter selection method is introduced, for the reference frame, all encoding and decoding processes must be completed before the encoding and decoding of the next frame can be started, which will significantly slow down the encoding speed.

[0086] Considering that the adjustment operation process of the deblocking filter is a fixed process, the subjective and objective optimization effects after filtering are limited. Therefore, the present disclosure performs adaptive optimization adjustment on the fixed process to improve the subjective and objective effects of the filtering tool.

[0087] Considering that the minimum granularity of parameter adjustment is at the slice level, that is, the same parameters are used for different content blocks in a slice and frame parallel processing cannot be achieved, resulting in low encoding and decoding efficiency, the present disclosure proposes a decision method to adaptively adjust parameters without increasing the bit rate transmission, taking into account the multi-frame parallel process of commercial codecs to achieve higher encoding and decoding performance. The embodiments of the present disclosure will be described in detail below.

[0088] Figure 5 is an overall flow chart of deblocking filtering according to some embodiments of the present disclosure.

[0089] Reference Figure 5 , a reconstructed block of the block is obtained by encoding the block (such as CTU), and at least one of the reconstructed block, the extended reconstructed block extended based on the reconstructed block, the feature of the reconstructed block and the feature of the extended reconstructed block is input into the neural network model, and filtering parameters for the reconstructed block can be obtained, such as at least one of the filter switch decision threshold beta, the pixel difference threshold Tc, the weak filtering decision value and the strong filtering decision value. After the neural network model predicts the filtering parameters, the predicted filtering parameters can be used according to Figure 4 The filtering process shown performs filtering processing and uses the filtered result for subsequent encoding.

[0090] In addition, in order to ensure the best rate-distortion performance, a transmission syntax element can be selected to inform the decoder whether to use the result predicted by the neural network model, that is, to make a rate-distortion RD decision. Figure 6 and Figure 7 The process of predicting filtering parameters and RD decision by the neural network model disclosed in the present invention is described in detail.

[0091] Figure 6 is a flowchart of a video encoding method according to some embodiments of the present disclosure. Figure 6 The method shown can be Figure 1 Video encoder 20 is shown performing.

[0092] Reference Figure 6 In step 701, a reconstructed image for the current frame is obtained. For example, Figure 2 The process described above obtains the reconstructed image of the current frame.

[0093] In step 702, based on the first-level block of the reconstructed image, a deblocking filter parameter for the first-level block is obtained using a neural network. In the present disclosure, the first-level block may be a strip, a tile, a coding tree unit, or a coding unit. The deblocking filter parameter may include at least one of a filter switch decision threshold, a pixel difference threshold, a weak filtering decision value, and a strong filtering decision value. For example, the filter switch decision threshold may be used to decide whether to turn on the filter switch, the pixel difference threshold may be used to determine the filtering strength, and the weak filtering decision value may be used to further adjust the weak filtering process in the deblocking filter. The strong filtering decision value may be used to further adjust the strong filtering process in the deblocking filter. Here, the weak filtering decision value may include decision values ​​for luminance short tap weak filtering and chrominance weak filtering, respectively, and the strong filtering decision value may include decision values ​​for luminance long filtering operation, luminance short tap strong filtering, and chrominance strong filtering, respectively. The output of the neural network may be set differently according to demand.

[0094] As an example, a deblocking filter parameter of the first-level block may be obtained using a neural network based on at least one of a coding feature of the first-level block and a reconstructed block obtained by expanding the first-level block.

[0095] Taking the first-level block as CTU as an example, the multi-dimensional features of CTU can be input into the neural network to obtain the deblocking filter parameters for the CTU. For example, the multi-dimensional features may include the quantization parameter QP, variance, prediction mode, block boundary features, etc. of the CTU. In this case, the neural network can be implemented by a multi-layer perceptron MLP.

[0096] For another example, the multi-dimensional features of the CTU and the reconstruction block based on the CTU expansion can be input into the neural network to obtain the deblocking filter parameters for the CTU. Assuming that the size of the CTU is N×N, M pixels are expanded outward on the basis of the CTU, that is, an expanded reconstruction block (such as an expanded brightness reconstruction block) is obtained, and its size is (N+M)×(N+M), where M can be greater than or equal to 8. At this time, the neural network can be implemented by a convolutional neural network and a multi-layer perceptron.

[0097] The above examples are only exemplary, and the extended reconstruction block size and neural network structure are not limited thereto. In addition, the input of the neural network can also be multiple first-level blocks and / or corresponding extended reconstruction blocks, so that the deblocking filter parameters of multiple first-level blocks can be obtained simultaneously.

[0098] The deblocking filter parameters of the first-level blocks are predicted through a neural network, so that each first-level block has a deblocking filter parameter suitable for itself, thereby improving the encoding and decoding performance.

[0099] In step 703, deblocking filtering is performed on the first-level block using the deblocking filtering parameters of the first-level block.

[0100] After obtaining the predicted deblocking filter parameters, Figure 4 The flow shown performs deblocking filtering.

[0101] For example, when it is determined that a luminance short tap weak filtering process is performed on the first-level block, the luminance short tap weak filtering process can be further adjusted based on the weak filtering decision value predicted by the neural network. As described above, in the current VVC standard, the luminance short tap weak filtering process is solidified into two steps, and the present disclosure further decides on this basis whether the first-level block is filtered according to the original process, only the first step of the original process (i.e., only p0 and q0 are corrected), or no weak filtering is performed.

[0102] For example, when following Figure 4 The process shown determines that when performing luminance short-tap weak filtering for the first-level block, when the weak filtering decision value is the first value, two rows of pixels on both sides of the boundary of the first-level block can be corrected; when the weak filtering decision value is the second value, one row of pixels on both sides of the boundary of the first-level block can be corrected; when the weak filtering decision value is the third value, two rows of pixels on both sides of the boundary of the first-level block may not be corrected. The first value may be 1, the second value may be 2, and the third value may be 3. The above examples are merely exemplary, and the present disclosure is not limited thereto.

[0103] The luminance long filtering operation process, luminance short tap strong filtering process, chrominance strong filtering and chrominance weak filtering in the deblocking filter can be changed in a similar manner as described above. In this case, in addition to predicting beta and Tc, the neural network of the present disclosure can also predict the decision values ​​for the luminance long filtering operation, luminance short tap strong filtering, luminance short tap weak filtering, chrominance strong filtering and chrominance weak filtering, respectively.

[0104] The coding efficiency is improved by using the weak filtering decision value to modify the inherent weak filtering decision process and using the strong filtering decision value to modify the inherent strong filtering decision process.

[0105] According to another embodiment of the present disclosure, after the deblocking filter parameters of the first-level block are derived using the neural network model, in order to ensure the optimal rate-distortion performance, a transmission flag can be selected to notify the decoding end whether to adopt the result predicted by the model.

[0106] For each first-level block, the first distortion information between the first filtering result obtained after deblocking filtering is performed on the first-level block using the deblocking filtering parameters predicted by the model and the original pixels of the first-level block can be calculated; the second distortion information between the second filtering result obtained after deblocking filtering is performed on the first-level block using the default filtering parameters and the original pixels of the first-level block can be calculated; based on the first distortion information and the second distortion information of the first-level block, a first syntax element for indicating whether to perform deblocking filtering on the first-level block using the deblocking filtering parameters predicted by the model is determined.

[0107] For example, taking the first-level block as CTU, the first syntax element can be designed as ctu_use_model_flag, and the entropy coding process with context in the VVC standard is used to encode the first syntax element. For the current CTU, the result of the parameter filtering predicted by the model is used to calculate the distortion (such as the square error SSE) between it and the original pixel, which is recorded as D model ; Calculate the distortion between the result after filtering with default parameters and the original pixel, marked as D default , where the default parameters can be the deblocking filter parameters determined according to the current VVC standard. Next, compare D default and D model , select the parameter corresponding to the minimum distortion as the parameter for deblocking filtering the current CTU. model <D default , then select the parameters predicted by the model and set ctu_use_model_flag to 1. If D model >D default , then choose to use the default parameters and set ctu_use_model_flag to 0. If D model =D default , it means that the distortion of the two is the same, and ctu_use_model_flag can be set to 0 or 1.

[0108] The finally determined CTU-level filtering parameters may be used for CTU filtering, and the filtered result may be used for subsequent encoding.

[0109] After determining the first syntax element of the first-level block, the encoder can send the first syntax element of the first-level block to the decoder using a signal, so that the decoder can know whether it is necessary to use a neural network to predict deblocking filter parameters for the first-level block and perform deblocking filtering based on this.

[0110] According to yet another embodiment of the present disclosure, a higher-level syntax element may be designed for a second-level block higher than the first-level block to further control encoding, decoding and signaling transmission of the first syntax element.

[0111] Taking the first-level block as CTU and the second-level block as a slice as an example, a second syntax element for indicating whether to enable the use of neural network prediction deblocking filter parameters for the second-level block and / or a third syntax element for indicating whether to enable the selection of deblocking filter parameters for the blocks in the second-level block can be designed at the slice level for secondary judgment. In other words, the second syntax element for the current slice can control whether all CTUs under the current slice enable the use of neural network prediction deblocking filter parameters as a whole. For example, the second syntax element can be designed as slice_use_model_flag and the third syntax element can be designed as ctu_adapt_flag. Here, the third syntax element can be used to set the value of the first syntax element to save signaling overhead when the first distortion information and the second distortion information of the first syntax element of the first-level block are the same.

[0112] Based on the first distortion information and the second distortion information of each block in the second-level blocks, it may be determined to use the second syntax element and / or the third syntax element and whether to encode and signal the first syntax element.

[0113] As an example, in a case where the first distortion information of each block in the second-level blocks including the first-level blocks is all greater than the second distortion information or the first distortion information is all less than the second distortion information, the first syntax element of each block may not be encoded and the first syntax element of each block may not be signaled; in a case where there is at least one block in the second-level blocks whose first distortion information is equal to the second distortion information, the first syntax element of each block may be encoded and the first syntax element of each block may be signaled.

[0114] In the case where the first distortion information of each block in the second-level block including the first-level block is all greater than the second distortion information, the value of the second syntax element may be set to 1 and the value of the third syntax element may be set to 0, indicating that the model-predicted parameters are used for filtering for each block in the second-level block and there is no need to select whether to use the model-predicted parameters or the default parameters. In this case, the first syntax element of each block may not be encoded and the first syntax element of each block may not be signaled and / or the third syntax element may not be encoded and signaled. Instead, the value of the first syntax element is derived as 1 and / or the third syntax element is derived as 0 by default at the decoding end. In the case where the first distortion information of each block is all less than the second distortion information, the value of the second syntax element may be set to 0 and the value of the third syntax element may be set to 0, indicating that the default parameters are used for filtering for each block in the second-level block. And there is no need to select whether to use model predicted parameters or default parameters. In this case, the first syntax element of each block may not be encoded and the first syntax element of each block may not be sent by signal and / or the third syntax element may not be encoded and signaled. Instead, the value of the first syntax element is derived as 0 by default and / or the third syntax element is derived as 0 at the decoding end. In the case where there is at least one block in the second-level block whose first distortion information is equal to the second distortion information, the value of the second syntax element may be set to 1 and the value of the third syntax element may be set to 1, indicating that model predicted parameters or default parameters are used for filtering for each block in the second-level block and it is necessary to select whether to use model predicted parameters or default parameters. In this case, the first syntax element of each block may be encoded and the first syntax element of each block may be sent by signal so that the decoding end can know the value of the first syntax element.

[0115] In the case where the first distortion information of at least one block in the second-level block is equal to the second distortion information, the value of the first syntax element may be 1 or 0. In this case, the value of the first syntax element of at least one block may be determined based on the codeword length of the first syntax element for each block in the second-level block.

[0116] For example, for the current stripe, the number of CTUs filtered using the model-predicted parameters, the number of CTUs filtered using the default parameters, and the number of CTUs filtered using both the model-predicted parameters and the default parameters can be counted among all CTUs in the current stripe, which are recorded as C model , C default , C same .

[0117] If C model=0, slice_use_model_flag=0, the ctu_use_model_flag of all CTUs under the slice is no longer encoded, and the default derivation is 0, and the ctu_adapt_flag is no longer encoded, and the default derivation is 0.

[0118] If C default =0, slice_use_model_flag=1, the ctu_use_model_flag of all CTUs under the slice is no longer encoded, and the default derivation is 1, and the ctu_adapt_flag is no longer encoded, and the default derivation is 0.

[0119] Will be with C same The corresponding ctu_use_model_flag is set to 0 and 1 respectively, and the ctu_use_model_flag of all CTUs under the slice is entropy coded to determine two situations (i.e., in the case of same The corresponding ctu_use_model_flag is set to 0 and in the case of same The corresponding ctu_use_model_flag is set to 1) to select the codeword length with C same The corresponding ctu_use_model_flag value is 1 or 0. In this case, slice_use_model_flag = 1, and ctu_adapt_flag = 1, indicating that the ctu_use_model_flag of all CTUs under the slice needs to be encoded and the ctu_use_model_flag value needs to be selected as 1 or 0 for the CTU (i.e., the CTU with the first loss information equal to the second loss information), and sent to the decoding end with a signal.

[0120] According to the above decision results, slice_use_model_flag, ctu_use_model_flag, and ctu_adapt_flag are entropy encoded.

[0121] Figure 7 is a flowchart of a video decoding method according to some embodiments of the present disclosure. Figure 7 The method shown can be Figure 1 The video decoder 30 shown performs.

[0122] Reference Figure 7 In step 801, a bit stream is received and a reconstructed image for a current frame is obtained based on the bit stream. Figure 3 The process shown obtains the reconstructed image.

[0123] In step 802, based on the first level block of the reconstructed image, a deblocking filter parameter for the first level block is obtained using a neural network. The first level block may be a strip, a tile, a coding tree unit, or a coding unit. The deblocking filter parameter may include at least one of a filter switch decision threshold, a pixel difference threshold, and a weak filtering decision value, wherein the weak filtering decision value is used to adjust the luminance short tap weak filtering process in the deblocking filter.

[0124] A first block level deblocking filter parameter may be obtained using a neural network based on at least one of a coding feature of the first level block and a reconstructed block obtained by expanding the first level block.

[0125] According to an embodiment of the present disclosure, a first syntax element for indicating whether to perform deblocking filtering on the first-level block using deblocking filtering parameters predicted by a model may be obtained from a bitstream. When the value of the first syntax element is 1, the deblocking filtering parameters for the first-level block may be obtained using a neural network based on the first-level block; when the value of the first syntax element is 0, the deblocking filtering parameters may not be obtained using a neural network, but default parameters may be used.

[0126] According to another embodiment of the present disclosure, a second syntax element for indicating whether to enable the use of a neural network to predict deblocking filter parameters for a second-level block including a first-level block and / or a third syntax element for indicating whether to enable the selection of a model-predicted deblocking filter parameter for a block of the second-level block can be obtained from a bitstream.

[0127] In the case where the second syntax element is 0 and the value of the third syntax element is 0 or the third syntax element is not obtained, the value of the first syntax element of each block in the second-level block may be derived to be 0. In the case where the second syntax element is 1 and the value of the third syntax element is 0 or the third syntax element is not obtained, the value of the first syntax element of each block may be derived to be 1. In the case where the values ​​of the second syntax element and the third syntax element are both 1, the first syntax element of each block is obtained from the bitstream.

[0128] In step 803, deblocking filtering is performed on the first-level block using the deblocking filtering parameters of the first block level. Figure 4 The process shown is combined with the weak filtering decision value of the present disclosure to perform deblocking filtering. For example, when determining to perform the luminance short tap weak filtering process, when the weak filtering decision value is the first value, two rows of pixels on both sides of the boundary of the first-level block are corrected; when the weak filtering decision value is the second value, one row of pixels on both sides of the boundary of the first-level block is corrected; when the weak filtering decision value is the third value, the two rows of pixels on both sides of the boundary of the first-level block are not corrected.

[0129] By adaptively optimizing and adjusting the solidified filtering process, the subjective and objective optimization effects after filtering can be improved.

[0130] By optimizing the granularity of filter parameter selection, frame parallel processing can be achieved, thereby improving encoding and decoding efficiency.

[0131] In addition, the above method can be implemented using an apparatus including one or more circuits, including an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components. The apparatus can use the circuit in combination with other hardware or software components to perform the above method. Each module, submodule, unit or subunit disclosed above can be implemented at least in part using one or more circuits.

[0132] Figure 8 A computing environment 1610 is shown coupled to a user interface 1650. The computing environment 1610 may be part of a data processing server. The computing environment 1610 includes a processor 1620, a memory 1630, and an input / output (I / O) interface 1640.

[0133] The processor 1620 generally controls the overall operation of the computing environment 1610, such as operations associated with display, data acquisition, data communication, and image processing. The processor 1620 may include one or more processors for executing instructions to perform all or some of the steps in the above method. In addition, the processor 1620 may include one or more modules that facilitate the interaction between the processor 1620 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0134] The memory 1630 is configured to store various types of data to support the operation of the computing environment 1610. The memory 1630 may include predetermined software 1632. Examples of such data include instructions for any application or method operating on the computing environment 1610, video data sets, image data, etc. The memory 1630 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0135] I / O interface 1640 provides an interface between processor 1620 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 1640 may be coupled to an encoder and a decoder.

[0136] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 1630 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 1620 in a computing environment 1610 to perform the above method. In one example, the plurality of programs can be executed by a processor 1620 in a computing environment 1610 to (for example, from Figure 2 The video encoder 20 in the computing environment 1610 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 1620 in the computing environment 1610 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 1620 in the computing environment 1610 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 1620 in the computing environment 1610 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0137] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0138] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 1620); and a non-transitory computer-readable storage medium or memory 1630 in which multiple programs that can be executed by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0139] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes the encoded video information generated by the above encoding method or the encoded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 1630 is also provided, wherein the plurality of programs can be executed by a processor 1620 in a computing environment 1610 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0140] In an embodiment, the computing environment 1610 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0141] In an embodiment, a method for storing a bit stream is also provided, comprising: storing the bit stream on a digital storage medium, wherein the bit stream comprises encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0142] In an embodiment, a method for transmitting a bit stream generated by the above encoder is also provided. In an embodiment, a method for receiving a bit stream to be decoded by the above decoder is also provided.

[0143] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0144] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0145] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video encoding method, comprising: Obtain a reconstructed image for the current frame; Based on the first-level block of the reconstructed image, using a neural network to obtain a deblocking filter parameter for the first-level block; as well as Deblocking filtering is performed on the first level block using the deblocking filtering parameters.

2. The method according to claim 1, wherein: Based on the first-level block of the reconstructed image, using a neural network to obtain a deblocking filter parameter for the first-level block, comprising: The deblocking filter parameters are obtained using the neural network based on at least one of the coding features of the first-level block and a reconstructed block obtained by expanding the first-level block.

3. The method according to claim 1, wherein: The deblocking filter parameters include at least one of a filter switch decision threshold, a pixel difference threshold, a weak filter decision value, and a strong filter decision value. The weak filtering decision value is used to further adjust the weak filtering process in the deblocking filter, and the strong filtering decision value is used to further adjust the strong filtering process in the deblocking filter.

4. The method according to claim 1, wherein: The first-level block is a slice, a tile, a coding tree unit or a coding unit.

5. The method according to claim 1, further comprising: Calculating first distortion information between a first filtering result obtained by performing deblocking filtering on the first-level block using the deblocking filtering parameters and original pixels of the first-level block; Calculating second distortion information between a second filtering result obtained by performing deblocking filtering on the first-level block using default filtering parameters and original pixels of the first-level block; Based on the first distortion information and the second distortion information of the first-level block, a first syntax element for indicating whether to perform deblocking filtering on the first-level block using the deblocking filtering parameters is determined.

6. The method according to claim 5, further comprising: determining the first distortion information and the second distortion information for each block in a second stage block including the first stage block; Based on the first distortion information and the second distortion information of each block, determine a second syntax element for indicating whether to use the neural network to predict the deblocking filter parameters for the second-level blocks and / or a third syntax element for indicating whether to select the deblocking filter parameters for the blocks in the second-level blocks.

7. The method according to claim 5 or 6, further comprising: In a case where all the first distortion information of each block in the second-level blocks including the first-level blocks is greater than the second distortion information or all the first distortion information is less than the second distortion information, the first syntax element of each block is not encoded and the first syntax element of each block is not signaled; In the case that there is at least one block in the second-level blocks whose first distortion information is equal to the second distortion information, the first syntax element of each block is encoded and the first syntax element of each block is transmitted by signaling.

8. The method according to claim 7, further comprising: When there is at least one block in the second-level blocks whose first distortion information is equal to the second distortion information, a value of the first syntax element of the at least one block is determined based on a codeword length of the first syntax element for each block.

9. The method according to claim 5 or 6, further comprising: In a case where the first distortion information of each block in the second-level block including the first-level block is all greater than the second distortion information, determining that the value of the second syntax element is 1 and the value of the third syntax element is 0; In a case where the first distortion information of each block is all smaller than the second distortion information, determining that a value of the second syntax element is 0 and a value of the third syntax element is 0; In the case that there is at least one block in the second-level blocks whose first distortion information is equal to the second distortion information, it is determined that the value of the second syntax element is 1 and the value of the third syntax element is 1.

10. The method according to claim 3, wherein: Performing deblocking filtering on the first-level block using the deblocking filtering parameters includes: Adjust the brightness short tap weak filtering process based on the weak filtering decision value, Wherein, when the weak filtering decision value is the first value, two rows of pixels on both sides of the boundary of the first-level block are corrected; When the weak filtering decision value is the second value, correcting one row of pixels on both sides of the boundary of the first-level block; When the weak filtering decision value is the third value, two rows of pixels on both sides of the boundary of the first-level block are not corrected.

11. A video decoding method, comprising: receiving a bit stream and obtaining a reconstructed image for a current frame based on the bit stream; Based on the first-level block of the reconstructed image, using a neural network to obtain a deblocking filter parameter for the first-level block; as well as Deblocking filtering is performed on the first level block using the deblocking filtering parameters.

12. The method according to claim 11, wherein: Based on the first-level block of the reconstructed image, using a neural network to obtain a deblocking filter parameter for the first-level block, comprising: obtaining the deblocking filter parameters using the neural network based on at least one of the coding feature of the first-level block and a reconstructed block obtained by expanding the first-level block; or A first syntax element for indicating whether to use the deblocking filter parameters to perform deblocking filtering on the first-level block is obtained from the bitstream; when the value of the first syntax element is 1, the deblocking filter parameters for the first-level block are obtained using the neural network based on the first-level block; when the value of the first syntax element is 0, the deblocking filter parameters are not obtained using the neural network.

13. A device for video encoding and decoding, comprising: one or more processors; as well as a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, Wherein, when executing the instructions, the one or more processors are configured to perform the method according to any one of claims 1-12.

14. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the method according to any one of claims 1-10 to encode first to third syntax elements into a bitstream and send the bitstream; and / or cause the one or more computer processors to receive a bitstream and perform the method according to any one of claims 11-12 based on the bitstream.

15. A computer program product having instructions for storing a bitstream, wherein: The bitstream includes: Video data generated by the method of any one of claims 1 to 10; and / or Video data to be decoded by the method of any one of claims 11-12.

Images