Video coding method, apparatus, computer readable medium, and electronic device

Abstract

Embodiments of the present application provide a video coding method, device, computer readable medium and electronic device. The video decoding method comprises: obtaining a residual signal corresponding to a current block; adding the residual signal and a prediction signal corresponding to the current block to obtain a reconstructed signal corresponding to the current block; performing statistics on residual distribution information corresponding to the current block, and comparing a residual distribution threshold corresponding to the current block transmitted by an encoding end with the residual distribution information; and determining whether to perform loop filtering processing on the reconstructed signal according to a comparison result. In the coding process, the technical solution of the embodiments of the present application adaptively starts to perform loop filtering processing according to the size comparison relationship between the residual distribution information and the residual distribution threshold corresponding to each block in each image frame, thereby effectively reducing the decoding complexity while not excessively losing the loop filtering gain.

Description

Technical Field

[0001] This application relates to the fields of computer and communication technology, and more specifically, to a video encoding / decoding method, apparatus, computer-readable medium, and electronic device. Background Technology

[0002] In the field of video coding, traditional loop filters are commonly used to suppress distortion in reconstructed images and improve their quality. However, traditional loop filters are ineffective at reducing distortion. While neural network-based loop filters can bring considerable gains to video coding, they also introduce excessive complexity, severely hindering their application and widespread adoption in practical scenarios. Therefore, there is an urgent need to propose a video coding scheme that can effectively reduce decoding complexity. Summary of the Invention

[0003] Embodiments of this application provide a video encoding / decoding method, apparatus, computer-readable medium, electronic device, and computer-readable storage medium. During the encoding / decoding process, for each block in each image frame, based on the comparison between its corresponding residual distribution information and the residual distribution threshold, loop filtering is adaptively initiated, thereby effectively reducing decoding complexity without excessively sacrificing loop filtering gain.

[0004] In a first aspect, embodiments of this application provide a video decoding method, comprising: decoding to obtain a residual signal corresponding to a current block; adding the residual signal to a prediction signal corresponding to the current block to obtain a reconstructed signal corresponding to the current block; statistically analyzing the residual distribution information corresponding to the current block, and comparing the residual distribution threshold transmitted from the encoding end to the residual distribution information; and determining whether to perform loop filtering processing on the reconstructed signal based on the comparison result.

[0005] Secondly, embodiments of this application provide a video encoding method, comprising: obtaining a reconstructed image based on reconstruction processing of encoded image frames; predicting residual distribution thresholds corresponding to each block in the reconstructed image based on performing loop filtering processing on the reconstructed image; wherein the comparison result between the residual distribution information corresponding to each block and the residual distribution thresholds is used to determine whether to perform loop filtering processing on each block; and transmitting the residual distribution thresholds corresponding to each block to a decoding end.

[0006] Thirdly, embodiments of this application provide a video decoding apparatus, including: a decoding unit configured to decode to obtain a residual signal corresponding to a current block; a first reconstruction unit configured to add the residual signal to a prediction signal corresponding to the current block to obtain a reconstructed signal corresponding to the current block; and a filtering unit configured to perform statistical analysis on the residual distribution information corresponding to the current block, compare the residual distribution threshold transmitted from the encoding end to the residual distribution information, and determine whether to perform loop filtering on the reconstructed signal based on the comparison result.

[0007] Fourthly, embodiments of this application provide a video encoding apparatus, comprising: a second reconstruction unit configured to obtain a reconstructed image based on reconstruction processing of an encoded image frame; a prediction unit configured to predict residual distribution thresholds corresponding to each block in the reconstructed image based on performing loop filtering processing on the reconstructed image; wherein the comparison result between the residual distribution information corresponding to each block and the residual distribution threshold is used to determine whether to perform loop filtering processing on each block; and a transmission unit configured to transmit the residual distribution thresholds corresponding to each block to a decoding end.

[0008] Fifthly, embodiments of this application provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the video decoding method or video encoding method as described in the above embodiments.

[0009] Sixthly, embodiments of this application provide an electronic device, including: one or more processors; and a storage device for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the video decoding method or video encoding method as described in the above embodiments.

[0010] In a seventh aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of an electronic device reads from and executes the computer program from the computer-readable storage medium, causing the electronic device to perform the video decoding or video encoding methods provided in the various alternative embodiments described above.

[0011] Eighthly, embodiments of this application provide a method for processing video streams, which can be generated according to the video encoding method provided in the above embodiments, or the video stream can be decoded according to the video decoding method provided in the above embodiments.

[0012] Ninthly, embodiments of this application provide an apparatus for processing video streams, which can be generated according to the video encoding method provided in the above embodiments, or the video stream can be decoded according to the video decoding method provided in the above embodiments.

[0013] In some embodiments of this application, after the decoding end decodes the residual signal corresponding to the current block, it adds the residual signal to the prediction signal corresponding to the current block to obtain the reconstructed signal corresponding to the current block. Then, it performs statistical analysis on the residual distribution information corresponding to the current block and compares the statistically obtained residual distribution information with the residual distribution threshold transmitted by the encoding end. Based on the comparison result, it determines whether to perform loop filtering on the reconstructed signal corresponding to the current block. Thus, the decoding process adaptively initiates loop filtering based on the comparison between the residual distribution information and the residual distribution threshold corresponding to the current block, effectively reducing decoding complexity without excessively sacrificing loop filtering gain. On the other hand, the encoding process can also avoid determining whether to enable loop filtering by encoding a flag bit block by block, saving a certain number of bits consumed in encoding.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0015] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0016] Figure 2 This diagram illustrates the placement of the video encoding and decoding devices in a streaming system.

[0017] Figure 3 A basic flowchart of a video encoder is shown;

[0018] Figure 4 This demonstrates how to use a loop filter based on a neural network;

[0019] Figure 5 A flowchart of a video decoding method according to an embodiment of this application is shown;

[0020] Figure 6 An example of a block-partitioned image is shown;

[0021] Figure 7 A flowchart of a video encoding method according to an embodiment of this application is shown;

[0022] Figure 8 An example of image region segmentation in a reconstructed image is shown;

[0023] Figure 9 A block diagram of a video decoding apparatus according to an embodiment of this application is shown;

[0024] Figure 10 A block diagram of a video encoding apparatus according to an embodiment of this application is shown;

[0025] Figure 11 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0026] Exemplary embodiments will now be described in a more comprehensive manner with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to these examples; rather, these embodiments are provided so that this application will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0027] Furthermore, the features, structures, or characteristics described in this application can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to provide a full understanding of the embodiments of this application. However, those skilled in the art will recognize that when implementing the technical solutions of this application, not all the detailed features in the embodiments may be used, one or more specific details may be omitted, or other methods, elements, devices, steps, etc., may be employed.

[0028] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0029] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0031] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0032] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown.

[0033] like Figure 1 As shown, system architecture 100 includes multiple terminal devices that can communicate with each other via, for example, a network 150. For instance, system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected via network 150. Figure 1 In one embodiment, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0034] For example, the first terminal device 110 can encode video data (e.g., a video image stream captured by the terminal device 110) to transmit it to the second terminal device 120 via the network 150. The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device 120 can receive the encoded video data from the network 150, decode the encoded video data to recover the video data, and display video images based on the recovered video data.

[0035] In one embodiment of this application, system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded video data, such as during a video conference. For bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 may encode video data (e.g., a video image stream captured by the terminal device) for transmission over network 150 to the other terminal device. Each of the third terminal device 130 and the fourth terminal device 140 may also receive encoded video data transmitted by the other terminal device, decode the encoded video data to recover the video data, and display the video images on an accessible display device based on the recovered video data.

[0036] exist Figure 1In the embodiments shown, the first terminal device 110, the second terminal device 120, the third terminal device 130 and the fourth terminal device 140 may be servers or terminals, but the principles disclosed in this application are not limited to these.

[0037] Servers can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals can be smartphones, tablets, laptops, desktop computers, smart speakers, smart voice interaction devices, smartwatches, smart home appliances, in-vehicle terminals, aircraft, etc., but are not limited to these.

[0038] Figure 1 The network 150 shown represents any number of networks, including, for example, wired and / or wireless communication networks, that transmit encoded video data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140. The communication network 150 may exchange data in circuit-switched and / or packet-switched channels. This network may include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of network 150 may be irrelevant to the operation of the disclosure herein.

[0039] In one embodiment of this application, Figure 2 The illustration shows the placement of video encoding and decoding devices in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital television (TV), and storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0040] The streaming system may include an acquisition subsystem 213, which may include a video source 201 such as a digital camera, which creates an uncompressed video image stream 202. In an embodiment, the video image stream 202 includes samples captured by a digital camera. The video image stream 202 is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data 204 (or encoded video bitstream 204). The video image stream 202 may be processed by an electronic device 220, which includes a video encoding device 203 coupled to the video source 201. The video encoding device 203 may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video data 204 (or encoded video bitstream 204) is depicted as a thin line to emphasize the lower data volume of the encoded video data 204 (or encoded video bitstream 204), which may be stored on a streaming server 205 for future use. One or more streaming client subsystems, such as Figure 2 Client subsystems 206 and 208 can access streaming server 205 to retrieve copies 207 and 209 of encoded video data 204. Client subsystem 206 may include, for example, a video decoding device 210 in electronic device 230. Video decoding device 210 decodes the incoming copy 207 of the encoded video data and produces an output video picture stream 211 that can be displayed on display 212 (e.g., a screen) or another presentation device. In some streaming systems, the encoded video data 204, video data 207, and video data 209 (e.g., video stream) may be encoded according to certain video encoding / compression standards.

[0041] It should be noted that electronic devices 220 and 230 may include other components not shown in the figures. For example, electronic device 220 may include a video decoding device, and electronic device 230 may also include a video encoding device.

[0042] In one embodiment of this application, taking High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) from international video coding standards, as well as the Chinese national video coding standard AVS, as examples, after an input video frame image, the video frame image is divided into several non-overlapping processing units according to a block size. Each processing unit performs a similar compression operation. This processing unit is called a Coding Tree Unit (CTU) or Largest Coding Unit (LCU). The CTU can be further subdivided into more refined units to obtain one or more basic Coding Units (CUs). The CU is the most basic element in a coding process.

[0043] In another embodiment, this processing unit can also be called a tile, which is a rectangular area of ​​a multimedia data frame that can be independently decoded and encoded. In the Alliance for Open Media Video 1 (AV1) standard, the tile can be further subdivided into one or more superblocks (SBs). The SB is the starting point for block partitioning and can be further divided into multiple subblocks. The superblocks are then further subdivided into one or more blocks. Each block is the most basic element in a coding process. Optionally, an SB can contain several blocks (Bs).

[0044] The above method of dividing video frame images can be called block partition structure. The following introduces some concepts in the encoding process:

[0045] Predictive coding includes intra-frame prediction and inter-frame prediction. The original video signal is predicted from a selected reconstructed video signal to obtain a residual video signal. The encoder needs to decide which predictive coding mode to choose for the current coding unit (or coding block) and inform the decoder. Intra-frame prediction refers to the predicted signal coming from a region within the same image that has already been encoded and reconstructed; inter-frame prediction refers to the predicted signal coming from another encoded image (called a reference image) that is different from the current image.

[0046] Transform and Quantization: After the residual video signal undergoes transformation operations such as Discrete Fourier Transform (DFT) and Discrete Cosine Transform (DCT), the signal is transformed into the transform domain, and these are called transform coefficients. The transform coefficients are then subjected to lossy quantization, losing some information to make the quantized signal more suitable for compression. In some video coding standards, there may be more than one transform method to choose from. Therefore, the encoder needs to select one of the transform methods for the current coding unit (or coding block) and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually leads to greater distortion and a lower bit rate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually leads to less distortion and a higher bit rate.

[0047] Entropy coding, or statistical coding, involves statistically compressing the quantized transform-domain signal based on the frequency of each value, ultimately outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected coding mode and motion vector data, also requires entropy coding to reduce the bit rate. Statistical coding is a lossless coding method that effectively reduces the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) and Content-Adaptive Binary Arithmetic Coding (CABAC).

[0048] Context-Based Binary Arithmetic Coding (CABAC) primarily involves three steps: binarization, context modeling, and binary arithmetic coding. After binarizing the input syntax elements, the binary data can be encoded using either a regular coding mode or a bypass coding mode. The bypass coding mode eliminates the need to assign a specific probability model to each binary bit; the input binary bit bin value is directly encoded using a simple bypass encoder, thus accelerating the overall encoding and decoding speed. Generally, different syntax elements are not completely independent, and even identical syntax elements possess a certain degree of memory. Therefore, according to conditional entropy theory, using other encoded syntax elements for conditional coding can further improve coding performance compared to independent coding or memoryless coding. This encoded symbol information used as conditions is called the context. In the regular coding mode, the binary bits of the syntax elements sequentially enter the context modeler. The encoder assigns an appropriate probability model to each input binary bit based on the values ​​of previously encoded syntax elements or binary bits; this process is called context modeling. The context model corresponding to a grammatical element can be located using the context index increment (contextindexincrement, ctxIdxInc) and the context index start (contextindexStart, ctxIdxStart). After the bin value and the assigned probability model are fed into the binary arithmetic encoder for encoding, the context model needs to be updated based on the bin value, which is the adaptive process in encoding.

[0049] Loop Filtering: The transformed and quantized signal undergoes inverse quantization, inverse transform, and prediction compensation to obtain a reconstructed image. Due to the effects of quantization, the reconstructed image differs from the original image in some aspects, resulting in distortion. Therefore, filtering operations can be performed on the reconstructed image, such as deblocking filters (DB), sample adaptive offset (SAO), or adaptive loop filters (ALF), to effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent coded images to predict future image signals, the aforementioned filtering operations are also called loop filtering, i.e., filtering operations within the coding loop.

[0050] In one embodiment of this application, Figure 3 A basic flowchart of a video encoder is shown, illustrating the process using intra-frame prediction as an example. The original image signal s... k[x,y] and the predicted image signal Perform the difference operation to obtain the residual signal u. k [x,y], residual signal u k After transformation and quantization, [x,y] is obtained as quantization coefficients. These coefficients are then used to obtain the encoded bitstream through entropy coding, and to obtain the reconstructed residual signal u' through inverse quantization and inverse transform. k [x,y], predict image signal With the reconstructed residual signal u' k [x,y] superimposed to generate image signals Image signal On one hand, the signal is input to the intra-frame mode decision module and the intra-frame prediction unit for intra-frame prediction processing; on the other hand, the reconstructed image signal s' is output through loop filtering. k [x,y], reconstruct the image signal s' k [x,y] can be used as a reference image for the next frame for motion estimation and motion compensation prediction. Then, based on the result s' of the motion compensation prediction... r [x+m x ,y+m y ] and intra-frame prediction results Obtain the predicted image signal for the next frame. And continue repeating the above process until the coding is complete.

[0051] Based on the above encoding process, at the decoding end, for each encoding unit (or encoding block), after acquiring the compressed bitstream (i.e., bitstream), entropy decoding is performed to obtain various mode information and quantization coefficients. Then, the quantization coefficients undergo inverse quantization and inverse transform processing to obtain the residual signal. On the other hand, based on the known encoding mode information, the prediction signal corresponding to the encoding unit (or encoding block) can be obtained. Then, the residual signal and the prediction signal are added together to obtain the reconstructed signal. The reconstructed signal then undergoes loop filtering and other operations to generate the final output signal.

[0052] Existing hybrid coding frameworks use traditional loop filters to suppress distortion in reconstructed images, thereby improving image quality and aiming to restore the encoded reconstructed image to the original image. However, traditional loop filters are hand-designed and struggle to effectively reduce distortion, leaving significant room for optimization. Due to the superior performance of neural network tools in image processing, neural network loop filters (NNLF) have been applied to the loop filter module.

[0053] Figure 4This demonstrates how to use a neural network-based loop filter. The image to be filtered is input into a trained loop filter to obtain the filtered image. Neural network-based loop filters typically use a loss function to constrain the filtered image, aiming to restore it as closely as possible to the original image. The loss function measures the difference between the predicted and true values; a larger loss value indicates a greater difference, and the training objective is to reduce the loss value. Commonly used loss functions include, but are not limited to, the L1 norm loss function, the L2 norm loss function, and the Smooth L1 loss function.

[0054] While neural network-based loop filters can deliver substantial video coding gains, they also introduce excessive complexity, severely hindering their application and widespread adoption in practical scenarios. To address this issue, embodiments of this application propose that during the encoding and decoding process, for each block in an image frame, loop filtering is adaptively initiated based on a comparison between its corresponding residual distribution information and a residual distribution threshold. This effectively reduces decoding complexity without excessively sacrificing loop filtering gain.

[0055] The embodiments of this application will be described in detail below.

[0056] Figure 5 A flowchart of a video decoding method according to an embodiment of this application is shown. This video decoding method can be executed by a device with computing processing capabilities, such as a terminal device or a server. (Refer to...) Figure 5 As shown, this video decoding method includes at least S510 to S540, which are described in detail below:

[0057] S510 decodes to obtain the residual signal corresponding to the current block.

[0058] At the decoding end, for each block, after obtaining the compressed video bitstream, entropy decoding is first performed to obtain various mode information and quantized transform coefficients. Each transform coefficient undergoes inverse quantization and inverse transform processing to obtain the residual signal.

[0059] It should also be understood that the current block mentioned in this embodiment refers to the block that is currently being decoded.

[0060] S520: Add the residual signal corresponding to the current block to the prediction signal corresponding to the current block to obtain the reconstruction signal corresponding to the current block.

[0061] Based on the known encoding mode information, the decoder can obtain the prediction signal corresponding to each block. After adding the two signals together, the reconstructed signal can be obtained.

[0062] S530, perform statistics on the residual distribution information corresponding to the current block, and compare the residual distribution threshold corresponding to the current block transmitted by the encoding end with the residual distribution information.

[0063] The embodiments of this application take into account that the residual in video coding reflects the pixel differences between the predicted image and the original image. When the residual is small, the difference between the predicted image and the original image is small, resulting in higher quality reconstructed images and poorer loop filtering. When the residual is large, the difference between the predicted image and the original image is large, resulting in lower quality reconstructed images and better loop filtering. Therefore, whether to perform loop filtering processing on each block can be adaptively determined based on the residual information.

[0064] Residual distribution information is used to characterize the magnitude of the residuals. Residual distribution information includes, but is not limited to, residual pixel statistics and block partitioning statistics.

[0065] The residual pixel statistics aim to represent the magnitude of the residual through pixel differences, and are obtained by statistically analyzing the residual pixels of the current block. For example, the residual pixel value statistics may include at least one of the following:

[0066] The average pixel value corresponding to the residual signal;

[0067] The number of non-zero pixel values ​​contained in the residual signal;

[0068] The ratio of the number of non-zero pixel values ​​contained in the residual signal to the total number of pixels contained in the current block;

[0069] The number of zero pixel values ​​contained in the residual signal;

[0070] The ratio of the number of zero pixel values ​​contained in the residual signal to the total number of pixels contained in the current block.

[0071] Block partitioning statistics aim to represent the magnitude of residuals based on the block partitioning situation. Figure 6 An example of a block-divided image is shown, where white areas represent block boundaries and black areas represent pixel content that has not been divided into blocks. Figure 6 As can be seen, block partitioning can reflect the magnitude of the residual; that is, regions with dense block partitioning in a frame tend to have larger residuals, while regions with sparse block partitioning in a frame tend to have smaller residuals. Therefore, this embodiment uses block partitioning statistics to represent the magnitude of the residual.

[0072] Block partition statistics are obtained through decoding. For example, block partition statistics can be determined by at least one of the number, depth, and number of block partitions within the current block.

[0073] The residual distribution threshold is used to characterize the boundary between good and bad loop filtering performance. The residual distribution threshold is predicted and transmitted at the encoder. The prediction process of the residual distribution threshold is described in the subsequent coding embodiments and will not be repeated here.

[0074] After obtaining the residual distribution information corresponding to the current block, the decoder compares the residual distribution threshold transmitted from the encoder with this residual distribution information. Based on the comparison result, it determines whether to perform loop filtering on the reconstructed signal corresponding to the current block. Thus, by adaptively initiating loop filtering based on the comparison between the residual distribution information and the residual distribution threshold, the decoder can effectively reduce decoding complexity without excessively sacrificing loop filtering gain.

[0075] In some embodiments of this application, the residual distribution information corresponding to the current block is a residual pixel statistic, and the residual distribution threshold is a residual size threshold. The decoding end performs statistics on the residual pixel values ​​of the current block to obtain a residual pixel value statistic, and then compares the residual size threshold with the residual pixel value statistic to determine whether to perform loop filtering on the reconstructed signal based on the comparison result.

[0076] In some embodiments of this application, the residual distribution information corresponding to the current block is a block partitioning statistic, and the residual distribution threshold is a residual size threshold. After the decoder determines the block partitioning statistic corresponding to the current block, it compares the block partitioning statistic with the block partitioning threshold, and then determines whether to perform loop filtering on the reconstructed signal based on the comparison result.

[0077] S540 determines whether to perform loop filtering on the reconstructed signal based on the comparison results.

[0078] In some embodiments of this application, if the residual distribution information corresponding to the current block is less than the residual distribution threshold corresponding to the current block, it is determined that the reconstructed signal corresponding to the current block will not be subjected to loop filtering; otherwise, it is determined that the reconstructed signal corresponding to the current block will be subjected to loop filtering.

[0079] In some embodiments of this application, if the residual distribution information corresponding to the current block is greater than the residual distribution threshold corresponding to the current block, it is determined that the reconstructed signal corresponding to the current block will not be subjected to loop filtering; otherwise, it is determined that the reconstructed signal corresponding to the current block will be subjected to loop filtering.

[0080] However, it should be noted that regardless of which exemplary embodiment is used to determine whether to perform loop filtering on the reconstructed signal corresponding to the current block, the technical principle can be summarized as follows: when the residual is large, loop filtering is performed on the reconstructed signal corresponding to the current block to obtain a better loop filtering effect; when the residual is small, since the loop filtering effect is poor, loop filtering is not performed, thereby effectively reducing the decoding complexity without excessively losing loop filtering gain.

[0081] In some embodiments of this application, since the decoding end adaptively initiates loop filtering based on a comparison between the residual distribution information corresponding to the current block and the residual distribution threshold, the encoding process can avoid encoding the loop filtering flag bit block by block to determine whether to enable loop filtering, thus saving a certain number of bits consumed by encoding. Correspondingly, the decoding end also does not obtain the loop filtering flag bit corresponding to the current block from the compressed video bitstream.

[0082] In some embodiments of this application, the encoding end can still encode the loop filter flag bit block by block to determine whether to enable loop filtering. The decoding end obtains the loop filter flag bit corresponding to the current block from the compressed video bitstream, and determines whether to perform loop filtering processing on the reconstructed signal corresponding to the current block based on the comparison result and the loop filter flag bit.

[0083] Based on the comparison results and the loop filtering flag, the method for determining whether to perform loop filtering on the reconstructed signal corresponding to the current block may include at least one of the following:

[0084] The corresponding loop filter flag value is determined based on the comparison results, and the determined loop filter flag value is used to cover the loop filter flag bit. Based on the covered loop filter flag bit, it is determined whether to perform loop filtering processing on the reconstructed signal.

[0085] Remove the loop filter flag and determine whether to perform loop filtering on the reconstructed signal based on the comparison results;

[0086] The comparison results are overwritten based on the loop filter flag, and the reconstructed signal is then determined to be loop filtered based on the overwritten comparison results.

[0087] Remove the comparison results and determine whether to perform loop filtering on the reconstructed signal based on the loop filtering flag.

[0088] It should also be noted that the loop filtering process mentioned in this embodiment can be implemented by calling traditional loop filters such as Deblocking Filter (DBF), Adaptive Loop Filter (ALF), and Sample Adaptive Offset (SAO), or by calling neural network-based filters.

[0089] The video decoding method mentioned in this embodiment can be applied to at least one of frame level, block level, coding tree unit (CTU) level, prediction unit (PU) level, and transform unit (TU) level, and can also be applied to at least one of Y component, U component, and V component.

[0090] Figure 7 A flowchart of a video encoding method according to an embodiment of this application is shown. This video encoding method can be executed by a device with computing processing capabilities, such as a terminal device or a server. (Refer to...) Figure 7 As shown, this video encoding method includes at least S710 to S730, which are detailed below:

[0091] S710 obtains a reconstructed image based on the reconstruction processing of the encoded image frame.

[0092] During the encoding process, the encoded image undergoes inverse quantization, inverse transform, and prediction compensation to obtain a reconstructed image. Compared to the original image, the reconstructed image differs in some information due to the influence of quantization, resulting in distortion. Therefore, loop filtering is required for the reconstructed image.

[0093] S720, based on performing loop filtering on the reconstructed image, predicts the residual distribution threshold corresponding to each block in the reconstructed image; wherein, the comparison result between the residual distribution information corresponding to each block and the residual distribution threshold is used to determine whether to perform loop filtering on each block.

[0094] The comparison between the residual distribution information corresponding to each block in the reconstructed image and the residual distribution threshold determines whether to perform loop filtering on each block. Therefore, loop filtering performed on the reconstructed image based on different residual distribution thresholds will result in different filtering effects.

[0095] This can also be understood as the number of blocks in the filtered image that have undergone loop filtering being controlled by the residual distribution threshold. As a result, the number of blocks in the filtered image that have undergone loop filtering will be different depending on the residual distribution threshold used to perform loop filtering on the reconstructed image.

[0096] The encoding end performs loop filtering on the reconstructed image and predicts the residual distribution threshold corresponding to each block in the reconstructed image. This residual distribution threshold can be either the residual distribution threshold that enables the reconstructed image to obtain the optimal filtering effect, or the residual distribution threshold that enables the reconstructed image to obtain the filtering effect that reaches the preset filtering effect.

[0097] In some embodiments of this application, the process by which the decoding end predicts the residual distribution threshold corresponding to each block in the reconstructed image based on performing loop filtering on the reconstructed image includes the following steps:

[0098] S721, traverse the preset candidate list of residual distribution thresholds, and determine the number of blocks to filter the reconstructed image based on the traversed residual distribution thresholds. Perform loop filtering on the reconstructed image according to the number of blocks to obtain filtering distortion. Calculate the corresponding filtering cost based on the filtering distortion and the number of blocks.

[0099] S722, based on the filtering cost corresponding to each residual distribution threshold in the residual distribution threshold candidate list, select the target residual distribution threshold from the residual distribution threshold candidate list, and determine the residual distribution threshold corresponding to each block in the reconstructed image based on the target residual distribution threshold.

[0100] In the above process, the candidate list of residual distribution thresholds contains at least one preset residual distribution threshold. The at least one residual distribution threshold in the candidate list can be predetermined by analyzing the filtering performance of a pre-statistically counted number of video codecs.

[0101] For each residual distribution threshold encountered by iterating through the candidate threshold list, the process involves determining the number of blocks to filter the reconstructed image based on that threshold, then performing loop filtering on the reconstructed image according to that number of blocks to obtain filtering distortion, and finally calculating the corresponding filtering cost based on the filtering distortion and the number of blocks. This process predicts the filtering cost obtained by performing loop filtering on the reconstructed image using each residual distribution threshold. The filtering cost can be understood as a numerical measure of the filtering effect; a lower filtering cost indicates a better filtering effect, and a higher filtering cost indicates a worse filtering effect.

[0102] Determining the number of blocks to filter the reconstructed image based on a residual distribution threshold means comparing the residual distribution information of each block with the residual distribution threshold to determine whether to perform loop filtering on each block, thus obtaining the number of blocks to filter the reconstructed image. For example, if the residual distribution information is less than the residual distribution threshold, it is determined that loop filtering will not be performed; otherwise, loop filtering will be performed. Conversely, if the residual distribution information is greater than the residual distribution threshold, it is determined that loop filtering will not be performed; otherwise, loop filtering will be performed.

[0103] For example, suppose a reconstructed image contains 10 blocks with residual distribution information of 1, 2, 3, ... 10, and the residual distribution threshold candidate list contains residual distribution thresholds of 2, 5, 8, and 11. Then, when the residual distribution threshold is 2, one block in the reconstructed image has a residual distribution information less than 2. If it is determined that no loop filtering is performed when the residual distribution information is less than the residual distribution threshold, the number of blocks in the reconstructed image to be filtered is 9. Similarly, when the residual distribution threshold is 8, seven blocks in the reconstructed image have residual distribution information less than 8, and the number of blocks in the reconstructed image to be filtered is 3.

[0104] Filtering distortion refers to the pixel differences between the original image and the filtered image corresponding to the reconstructed image. For example, the mean absolute error (MAE) of pixels between the original and filtered images can be calculated to obtain the filtering distortion. Alternatively, the mean squared error (MSE) of pixels between the original and filtered images can also be calculated to obtain the filtering distortion.

[0105] In some embodiments of this application, the filtering cost can be obtained by calculating the sum of the filtering distortion and the number of blocks used to filter the reconstructed image.

[0106] In some embodiments of this application, a filtering factor can be determined based on the image features corresponding to the encoded image frame. Then, the product of the filtering factor and the number of blocks used to filter the reconstructed image is calculated, and the sum of this product and the filtering distortion is calculated to obtain the filtering cost. Image features include, but are not limited to, at least one of quantization parameters (QP), block size, and image resolution. It can also be understood that the filtering factor can be derived from information such as quantization parameters, block size, and image resolution.

[0107] This calculation process can be expressed as the following formula:

[0108] Cost = D + λ * N

[0109] Where Cost represents the filtering cost, D represents the filtering distortion, λ represents the filtering factor, and N represents the number of blocks used to filter the reconstructed image. Thus, the corresponding filtering cost is calculated for each residual distribution threshold in the candidate list of residual distribution thresholds.

[0110] In some embodiments of this application, the residual distribution threshold with the lowest filtering cost can be selected as the target residual distribution threshold, or other residual distribution thresholds can be selected from the residual distribution threshold candidate list as the target residual distribution threshold based on the filtering cost.

[0111] In some embodiments of this application, the target residual distribution threshold can be used as the residual distribution threshold corresponding to each block in the reconstructed image, or a preset calculation can be performed based on the target residual distribution threshold to obtain the residual distribution threshold corresponding to each block in the reconstructed image.

[0112] In some embodiments of this application, the process by which the decoding end predicts the residual distribution threshold corresponding to each block in the reconstructed image based on performing loop filtering on the reconstructed image includes the following steps:

[0113] S723, determine the number of blocks to filter the reconstructed image based on the initial residual distribution threshold, and perform loop filtering on the reconstructed image according to the number of blocks to obtain the filtering distortion, and calculate the corresponding filtering cost based on the filtering distortion and the number of blocks;

[0114] S724, iteratively execute the process of updating the residual distribution threshold according to the preset rules, performing loop filtering on the reconstructed image based on the updated residual distribution threshold, and obtaining the corresponding filtering cost, until the updated residual distribution threshold exceeds the preset residual range;

[0115] S725: Based on the filtering cost corresponding to each residual distribution threshold, select the target residual distribution threshold from each residual distribution threshold, and determine the residual distribution threshold corresponding to each block in the reconstructed image based on the target residual distribution threshold.

[0116] In the above process, an initial residual distribution threshold is preset, and the filtering cost corresponding to the loop filtering process of the reconstructed image is obtained. This initial residual distribution threshold can also be predetermined by analyzing the filtering performance of pre-statistically counted video codecs. Then, the process of iteratively updating the residual distribution threshold according to preset rules and obtaining the filtering cost corresponding to the loop filtering process of the reconstructed image using the updated residual distribution threshold is executed until the updated residual distribution threshold exceeds a preset residual range. The preset rules can be based on a preset value that increments or decrements, or other rules, and are not limited here. Finally, based on the filtering cost corresponding to each residual distribution threshold, a target residual distribution threshold is selected from the residual distribution thresholds, and the residual distribution threshold corresponding to each block in the reconstructed image is determined based on the target residual distribution threshold.

[0117] It should be noted that the determination of the number of blocks for filtering the reconstructed image, the acquisition of filtering distortion, the calculation of filtering cost, the method of selecting the target residual distribution threshold, and the method of finally determining the residual distribution threshold based on the target residual distribution threshold involved in this embodiment can all be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here.

[0118] S730 transmits the residual distribution thresholds corresponding to each block in the reconstructed image to the decoding end.

[0119] After predicting the residual distribution thresholds for each block in the reconstructed image, the encoder transmits these thresholds to the decoder. The decoder then performs the corresponding decoding processing for each block based on these thresholds. For details on how the decoder performs the decoding processing based on the residual distribution thresholds transmitted from the encoder, please refer to [link to documentation / reference]. Figure 5 The embodiments shown will not be described in detail here.

[0120] It should be noted that the encoding end does not need to transmit block-level loop filter flags to the decoding end to indicate the switching of the loop filters corresponding to each block again, thus saving the number of encoding bits. Of course, in some embodiments of this application, the encoding end can still transmit block-level loop filter flags to the decoding end, and the operation of the loop filter flags by the decoding end when performing decoding on each block is also described in the foregoing embodiments.

[0121] It should also be noted that the video coding method mentioned in this embodiment can be applied to at least one of frame level, block level, coding tree unit (CTU) level, prediction unit (PU) level, and transform unit (TU) level, and can also be applied to at least one of Y component, U component, and V component.

[0122] The loop filtering process mentioned in this embodiment can be implemented by calling traditional loop filters such as Deblocking Filter (DBF), Adaptive Loop Filter (ALF), and Sample Adaptive Offset (SAO), or by calling neural network-based filters.

[0123] The residual distribution information mentioned in this embodiment includes, but is not limited to, residual pixel statistics and block partitioning statistics. When the residual distribution information is a residual pixel value statistic, the residual distribution threshold is a residual size threshold. When the residual distribution information is a block partitioning statistic, the residual distribution threshold is a block partitioning threshold.

[0124] Block partitioning statistics can be determined by at least one of the number, depth, and number of block partitions within the current block.

[0125] Residual pixel value statistics may include at least one of the following:

[0126] The average pixel value corresponding to the residual signal;

[0127] The number of non-zero pixel values ​​contained in the residual signal;

[0128] The ratio of the number of non-zero pixel values ​​contained in the residual signal to the total number of pixels contained in the current block;

[0129] The number of zero pixel values ​​contained in the residual signal;

[0130] The ratio of the number of zero pixel values ​​contained in the residual signal to the total number of pixels contained in the current block.

[0131] It should also be noted that, in some embodiments of this application, when the reconstructed image contains at least two image regions, the residual distribution threshold corresponding to each image region is predicted respectively, and the residual distribution threshold corresponding to each block contained in the reconstructed image is determined based on the residual distribution threshold corresponding to each image region.

[0132] This embodiment does not limit the specific way the image regions are divided; the size of each image region can be the same or different. For example, Figure 8 An example of image region division for a reconstructed image is shown, in which the reconstructed image is divided into four image regions according to various division methods.

[0133] The process of predicting the residual distribution threshold for each image region in this embodiment is the same as the process of predicting the residual distribution threshold in S720, and will not be described in detail here.

[0134] After obtaining the residual distribution thresholds for each image region, the residual distribution threshold for each image region can be used as the residual distribution thresholds for each block contained within that image region. For example, assuming the residual distribution threshold for image region 1 is r1, the residual distribution threshold for image region 2 is r2, the residual distribution threshold for image region 3 is r3, and the residual distribution threshold for image region 4 is r4, then the residual distribution thresholds for each block contained in image region 1 are all r1, the residual distribution thresholds for each block contained in image region 2 are all r2, the residual distribution thresholds for each block contained in image region 3 are all r3, and the residual distribution thresholds for each block contained in image region 4 are all r4.

[0135] After obtaining the residual distribution thresholds corresponding to each image region, the frame-level residual distribution thresholds can also be determined based on these thresholds. These frame-level residual distribution thresholds can then be used as the residual distribution thresholds for each block within the reconstructed image. For example, the sum of the residual distribution thresholds for each image region can be used as the frame-level residual distribution threshold. Alternatively, a weighted sum or other calculations can be performed on the residual distribution thresholds for each image region to obtain the frame-level residual distribution thresholds.

[0136] Since the processing at the encoding end is similar to that at the decoding end, the specific details can be found in the aforementioned processing procedures at the decoding end, and will not be repeated in this embodiment.

[0137] The following describes an apparatus embodiment of this application, which can be used to perform the methods described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments described above.

[0138] Figure 9 A block diagram of a video decoding apparatus according to an embodiment of the present application is shown. The video decoding apparatus can be installed in a device with computing processing capabilities, such as a terminal device or a server.

[0139] Reference Figure 9 As shown, a video decoding apparatus 900 according to an embodiment of this application includes: a decoding unit 910, a first reconstruction unit 920, and a filtering unit 930.

[0140] The decoding unit 910 is configured to decode the residual signal corresponding to the current block.

[0141] The first reconstruction unit 920 is configured to add the residual signal to the prediction signal corresponding to the current block to obtain the reconstruction signal corresponding to the current block.

[0142] The filtering unit 930 is configured to perform statistical analysis on the residual distribution information corresponding to the current block, compare the residual distribution threshold corresponding to the current block transmitted from the encoding end with the residual distribution information, and determine whether to perform loop filtering on the reconstructed signal based on the comparison result.

[0143] In some embodiments of this application, based on the foregoing scheme, the residual distribution information includes residual pixel value statistics, and the residual distribution threshold includes a residual size threshold; the filtering unit 930 is further configured to perform the following steps:

[0144] The residual pixel values ​​of the current block are statistically analyzed to obtain the residual pixel value statistics.

[0145] Compare the residual size threshold with the residual pixel value statistics.

[0146] In some embodiments of this application, based on the foregoing scheme, the residual pixel value statistics include at least one of the following:

[0147] The average pixel value corresponding to the residual signal;

[0148] The number of non-zero pixel values ​​contained in the residual signal;

[0149] The ratio of the number of non-zero pixel values ​​to the total number of pixels contained in the current block;

[0150] The number of zero pixel values ​​contained in the residual signal;

[0151] The ratio of the number of zero pixel values ​​to the total number of pixels contained in the current block.

[0152] In some embodiments of this application, based on the foregoing scheme, the residual distribution information includes block partitioning statistics, and the residual distribution threshold includes a block partitioning threshold; the filtering unit 930 is further configured to perform the following steps:

[0153] Determine the block partition statistics corresponding to the current block;

[0154] Compare the block partitioning threshold with the block partitioning statistics.

[0155] In some embodiments of this application, based on the aforementioned scheme, the block partitioning statistics are determined by at least one of the number, depth, and number of block partitions within the current block.

[0156] In some embodiments of this application, based on the foregoing scheme, the filtering unit 930 is further configured to perform the following steps:

[0157] If the residual distribution information is less than the residual distribution threshold, it is determined that no loop filtering processing will be performed on the reconstructed signal; otherwise, it is determined that loop filtering processing will be performed on the reconstructed signal.

[0158] Alternatively, if the residual distribution information is greater than the residual distribution threshold, then it is determined that no loop filtering processing will be performed on the reconstructed signal; otherwise, it is determined that loop filtering processing will be performed on the reconstructed signal.

[0159] In some embodiments of this application, based on the foregoing scheme, the filtering unit 930 is further configured to perform the following steps:

[0160] Obtain the loop filter flag bit corresponding to the current block from the video bitstream. The loop filter flag bit is used to mark whether the current block has started loop filtering.

[0161] Based on the comparison results and the loop filtering flag, determine whether to perform loop filtering on the reconstructed signal.

[0162] In some embodiments of this application, based on the foregoing scheme, and based on the comparison results and the loop filter flag, the method for determining whether to perform loop filtering processing on the reconstructed signal includes at least one of the following:

[0163] The corresponding loop filter flag value is determined based on the comparison results, and the determined loop filter flag value is used to cover the loop filter flag bit. Based on the covered loop filter flag bit, it is determined whether to perform loop filtering processing on the reconstructed signal.

[0164] Remove the loop filter flag and determine whether to perform loop filtering on the reconstructed signal based on the comparison results;

[0165] The comparison results are overwritten based on the loop filter flag, and the reconstructed signal is then determined based on the overwritten comparison results.

[0166] Remove the comparison results and determine whether to perform loop filtering on the reconstructed signal based on the loop filtering flag.

[0167] In some embodiments of this application, based on the aforementioned scheme, loop filtering is performed by calling a neural network model.

[0168] In some embodiments of this application, based on the foregoing scheme, the video decoding device 900 can be applied to at least one of frame level, block level, coding tree unit level, prediction unit level, and transform unit level, or to at least one of Y component, U component, and V component.

[0169] The video decoding device and the video decoding method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the video decoding device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0170] Figure 10 A block diagram of a video encoding apparatus according to an embodiment of the present application is shown. The video encoding apparatus can be installed in a device with computing processing capabilities, such as a terminal device or a server.

[0171] Reference Figure 10 As shown, a video encoding apparatus 1000 according to an embodiment of this application includes: a second reconstruction unit 1010, a prediction unit 1020, and a transmission unit 1030.

[0172] The second reconstruction unit 1010 is configured to obtain a reconstructed image based on the reconstruction processing of the encoded image frame.

[0173] The prediction unit 1020 is configured to predict the residual distribution threshold corresponding to each block in the reconstructed image based on performing loop filtering on the reconstructed image; wherein, the comparison result between the residual distribution information corresponding to each block and the residual distribution threshold is used to determine whether to perform loop filtering on each block.

[0174] The transmission unit 1030 is configured to transmit the residual distribution threshold corresponding to each block to the decoding end.

[0175] In some embodiments of this application, based on the foregoing scheme, the prediction unit 1020 is further configured to perform the following steps:

[0176] Traverse the preset candidate list of residual distribution thresholds and determine the number of blocks to filter the reconstructed image based on the traversed residual distribution thresholds. Perform loop filtering on the reconstructed image according to the number of blocks to obtain the filtering distortion. Calculate the corresponding filtering cost based on the filtering distortion and the number of blocks.

[0177] Based on the filtering cost corresponding to each residual distribution threshold in the residual distribution threshold candidate list, a target residual distribution threshold is selected from the residual distribution threshold candidate list, and the residual distribution threshold corresponding to each block in the reconstructed image is determined based on the target residual distribution threshold.

[0178] In some embodiments of this application, based on the foregoing scheme, the prediction unit 1020 is further configured to perform the following steps:

[0179] The number of blocks to filter the reconstructed image is determined based on the initial residual distribution threshold, and loop filtering is performed on the reconstructed image according to the number of blocks to obtain the filtering distortion. The corresponding filtering cost is calculated based on the filtering distortion and the number of blocks.

[0180] The process involves iteratively updating the residual distribution threshold according to a preset rule, performing loop filtering on the reconstructed image based on the updated residual distribution threshold, and obtaining the corresponding filtering cost, until the updated residual distribution threshold exceeds the preset residual range.

[0181] Based on the filtering cost corresponding to each residual distribution threshold, a target residual distribution threshold is selected from each residual distribution threshold, and the residual distribution threshold corresponding to each block in the reconstructed image is determined based on the target residual distribution threshold.

[0182] In some embodiments of this application, based on the foregoing scheme, the prediction unit 1020 performs the process of calculating the corresponding filtering cost based on filter distortion and the number of blocks, including:

[0183] The filtering factor is determined based on the image features corresponding to the encoded image frame; the image features include at least one of the following: quantization parameters, block size, and image resolution.

[0184] Calculate the product of the filter factor and the number of blocks, and sum the filter distortion with the product to obtain the filter cost corresponding to the residual distribution threshold traversed.

[0185] In some embodiments of this application, based on the foregoing scheme, the prediction unit 1020 is further configured to perform the following steps:

[0186] When the reconstructed image contains at least two image regions, predict the residual distribution threshold for each image region separately.

[0187] The residual distribution threshold for each block is determined based on the residual distribution threshold corresponding to each image region.

[0188] In some embodiments of this application, based on the foregoing scheme, the prediction unit 1020 is further configured to perform the following steps:

[0189] The residual distribution threshold corresponding to each image region is used as the residual distribution threshold corresponding to each block contained in that image region.

[0190] Alternatively, based on the residual distribution threshold corresponding to each image region, a frame-level residual distribution threshold can be determined, and the frame-level residual distribution threshold can be used as the residual distribution threshold for each block.

[0191] In some embodiments of this application, based on the aforementioned scheme, the residual distribution information includes residual pixel value statistics, and the residual distribution threshold includes a residual size threshold;

[0192] Alternatively, residual distribution information may include block partition statistics, and residual distribution thresholds may include block partition thresholds.

[0193] In some embodiments of this application, based on the foregoing scheme, the video encoding device 1000 can be applied to at least one of frame level, block level, coding tree unit level, prediction unit level, and transform unit level, or to at least one of Y component, U component, and V component.

[0194] The video encoding apparatus and the video encoding method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs its operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the video encoding apparatus provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0195] This application also provides a method and apparatus for processing video streams, wherein the video stream can be generated according to the video encoding method provided in the above embodiments, or the video stream can be decoded according to the video decoding method provided in the above embodiments.

[0196] Figure 11 A schematic diagram of a computer system suitable for implementing an electronic device according to the embodiments of this application is shown. The electronic device may be a video encoding device or a video decoding device as described in the foregoing embodiments.

[0197] It should be noted that, Figure 11 The computer system 1100 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0198] like Figure 11As shown, the computer system 1100 may include a central processing unit (CPU) 1101, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1102 or programs loaded from storage portion 1108 into random access memory (RAM) 1103, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1103. The CPU 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. An input / output (I / O) interface 1105 is also connected to bus 1104.

[0199] The following components can be connected to I / O interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to I / O interface 1105 as needed. Removable media 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1110 as needed so that computer programs read from them can be installed into storage section 1108 as needed.

[0200] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 1111. When the computer program is executed by central processing unit (CPU) 1101, it performs various functions defined in the system of this application.

[0201] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0202] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and a computer program.

[0203] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0204] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0205] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0206] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, and includes several instructions to cause an electronic device to execute the method according to the embodiments of this application.

[0207] For example, an electronic device can be a video decoding device, then the video decoding device can perform... Figure 5 The video decoding method shown; for example, an electronic device can be a video encoding device, then the video encoding device can perform... Figure 7 The video encoding method shown.

[0208] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0209] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A video decoding method, characterized in that, The method includes: Decoding yields the residual signal corresponding to the current block; The residual signal is added to the prediction signal corresponding to the current block to obtain the reconstruction signal corresponding to the current block; The residual distribution information corresponding to the current block is statistically analyzed, and the residual distribution threshold corresponding to the current block transmitted from the encoding end is compared with the residual distribution information. Based on the comparison results, determine whether to perform loop filtering on the reconstructed signal.

2. The method according to claim 1, characterized in that, The residual distribution information includes residual pixel value statistics, and the residual distribution threshold includes a residual size threshold; The step of statistically analyzing the residual distribution information corresponding to the current block and comparing the residual distribution threshold transmitted from the encoding end with the residual distribution information includes: The residual pixel values ​​of the current block are statistically analyzed to obtain the residual pixel value statistics. The residual size threshold is compared with the residual pixel value statistics.

3. The method according to claim 2, characterized in that, The residual pixel value statistics include at least one of the following: The average pixel value corresponding to the residual signal; The number of non-zero pixel values ​​contained in the residual signal; The ratio of the number of non-zero pixel values ​​to the total number of pixels contained in the current block; The number of zero pixel values ​​contained in the residual signal; The ratio of the number of zero pixel values ​​to the total number of pixels contained in the current block.

4. The method according to claim 1, characterized in that, The residual distribution information includes block partition statistics, and the residual distribution threshold includes a block partition threshold. The step of statistically analyzing the residual distribution information corresponding to the current block and comparing the residual distribution threshold transmitted from the encoding end with the residual distribution information includes: Determine the block partitioning statistics corresponding to the current block; The block partitioning threshold is compared with the block partitioning statistics.

5. The method according to claim 4, characterized in that, The block partitioning statistics are determined by at least one of the number, depth, and number of block partitions within the current block.

6. The method according to any one of claims 1-5, characterized in that, The step of determining whether to perform loop filtering on the reconstructed signal based on the comparison result includes: If the residual distribution information is less than the residual distribution threshold, it is determined that the reconstructed signal will not be subjected to loop filtering; otherwise, it is determined that the reconstructed signal will be subjected to loop filtering. or, If the residual distribution information is greater than the residual distribution threshold, it is determined that the reconstructed signal will not be subjected to loop filtering; otherwise, it is determined that the reconstructed signal will be subjected to loop filtering.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Obtain the loop filter flag bit corresponding to the current block from the video bitstream. The loop filter flag bit is used to mark whether the current block starts loop filtering. Based on the comparison results and the loop filter flag, it is determined whether to perform loop filtering on the reconstructed signal.

8. The method according to claim 7, characterized in that, Based on the comparison result and the loop filter flag, the method for determining whether to perform loop filtering on the reconstructed signal includes at least one of the following: Based on the comparison result, a corresponding loop filter flag value is determined, and the determined loop filter flag value is used to cover the loop filter flag bit, so as to determine whether to perform loop filter processing on the reconstructed signal based on the covered loop filter flag bit; Remove the loop filter flag and determine whether to perform loop filtering on the reconstructed signal based on the comparison result; The comparison result is overwritten based on the loop filter flag, so as to determine whether to perform loop filtering on the reconstructed signal based on the overwritten comparison result; Remove the comparison result and determine whether to perform loop filtering on the reconstructed signal based on the loop filtering flag.

9. The method according to any one of claims 1-8, characterized in that, The loop filtering process is performed by calling a neural network model.

10. A video encoding method, characterized in that, The method includes: Based on the reconstruction processing of the encoded image frames, a reconstructed image is obtained; Based on performing loop filtering on the reconstructed image, the residual distribution threshold corresponding to each block in the reconstructed image is predicted; wherein, the comparison result between the residual distribution information corresponding to each block and the residual distribution threshold is used to determine whether to perform loop filtering on each block; The residual distribution threshold corresponding to each block is transmitted to the decoding end.

11. The method according to claim 10, characterized in that, The step of predicting the residual distribution threshold corresponding to each block in the reconstructed image based on performing loop filtering processing on the reconstructed image includes: Traverse the preset candidate list of residual distribution thresholds, and determine the number of blocks to filter the reconstructed image based on the traversed residual distribution thresholds. Perform loop filtering on the reconstructed image according to the number of blocks to obtain filtering distortion. Calculate the corresponding filtering cost based on the filtering distortion and the number of blocks. Based on the filtering cost corresponding to each residual distribution threshold in the residual distribution threshold candidate list, a target residual distribution threshold is selected from the residual distribution threshold candidate list, and the residual distribution threshold corresponding to each block in the reconstructed image is determined based on the target residual threshold.

12. The method according to claim 10, characterized in that, The step of predicting the residual distribution threshold corresponding to each block in the reconstructed image based on performing loop filtering processing on the reconstructed image includes: The number of blocks to filter the reconstructed image is determined based on the initial residual distribution threshold, and loop filtering is performed on the reconstructed image according to the number of blocks to obtain filtering distortion. The corresponding filtering cost is calculated based on the filtering distortion and the number of blocks. The process involves iteratively updating the residual distribution threshold according to a preset rule, performing loop filtering on the reconstructed image based on the updated residual distribution threshold, and obtaining the corresponding filtering cost, until the updated residual distribution threshold exceeds the preset residual range. Based on the filtering cost corresponding to each residual distribution threshold, a target residual distribution threshold is selected from the residual distribution thresholds, and the residual distribution threshold corresponding to each block in the reconstructed image is determined based on the target residual distribution threshold.

13. The method according to claim 11 or 12, characterized in that, The process of calculating the corresponding filtering cost based on the filtering distortion and the number of blocks includes: The filtering factor is determined based on the image features corresponding to the encoded image frame; the image features include at least one of quantization parameters, block size, and image resolution. Calculate the product of the filtering factor and the number of blocks, and calculate the sum of the filtering distortion and the product to obtain the filtering cost corresponding to the traversed residual distribution threshold.

14. The method according to any one of claims 10-13, characterized in that, The method further includes: When the reconstructed image contains at least two image regions, predict the residual distribution threshold corresponding to each image region; The residual distribution threshold for each block is determined based on the residual distribution threshold corresponding to each image region.

15. The method according to claim 14, characterized in that, The step of determining the residual distribution threshold corresponding to each block based on the residual distribution threshold corresponding to each image region includes: The residual distribution threshold corresponding to each image region is used as the residual distribution threshold corresponding to each block contained in that image region; or, Based on the residual distribution threshold corresponding to each image region, a frame-level residual distribution threshold is determined, and the frame-level residual distribution threshold is used as the residual distribution threshold corresponding to each block.

16. A video decoding device, characterized in that, The device includes: The decoding unit is configured to decode the residual signal corresponding to the current block. The first reconstruction unit is configured to add the residual signal to the prediction signal corresponding to the current block to obtain the reconstruction signal corresponding to the current block; The filtering unit is configured to perform statistical analysis on the residual distribution information corresponding to the current block, compare the residual distribution threshold corresponding to the current block transmitted from the encoding end with the residual distribution information, and determine whether to perform loop filtering on the reconstructed signal based on the comparison result.

17. A video encoding device, characterized in that, The device includes: The second reconstruction unit is configured to obtain a reconstructed image based on the reconstruction processing of the encoded image frame; The prediction unit is configured to predict the residual distribution threshold corresponding to each block in the reconstructed image based on performing loop filtering on the reconstructed image; wherein, the comparison result between the residual distribution information corresponding to each block and the residual distribution threshold is used to determine whether to perform loop filtering on each block. The transmission unit is configured to transmit the residual distribution threshold corresponding to each block to the decoding end.

18. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-15.

19. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more computer programs that, when executed by one or more processors, cause the electronic device to perform the method of any one of claims 1-15.

Images