Time-domain Filtering Method, Device, Equipment and Medium Based on Video Block Error

Abstract

The present application discloses a time-domain filtering method, device, equipment and medium based on video block error, which relates to the technical field of audio and video coding. The method includes: determining the filtering weight corresponding to each video block according to the block error and block error distribution of the video block, improving the accuracy of filtering weight calculation in units of video blocks, and performing time-domain filtering on the coding frame to be filtered by integrating the filtering weights of the video blocks in adjacent frames. Thus, the intensity of time-domain filtering can be adjusted according to the block error and block error distribution, improving the time-domain filtering ability and eliminating or reducing the problem of performance loss of the filter.

Description

Technical Field

[0001] The present invention relates to the technical field of audio - video coding, and particularly to a temporal filtering method, device, equipment and medium based on video block error. Background Technique

[0002] Audio - video coding standards are technical specifications for compressing and encoding audio and video data, aiming to reduce the data volume for storage and transmission while ensuring a certain quality. Common audio - video coding standards include AVS3 (AudioVideo Standard3, the third - generation audio - video coding standard), H.265 / HEVC, H.266 / VVC, etc. An audio - video encoder is a software or hardware tool that implements the video compression algorithm of an audio - video coding standard, used to compress the original uncompressed video signal into a bitstream in a specific format for storage or transmission.

[0003] Temporal Filter (TF for short) is a common tool in various encoders. By filtering the image that is more likely to be referenced in inter - frame coding in the time domain (note: whether a certain frame is easy to be referenced is specified artificially before coding), the residual of the frame encoded with this image as a reference is reduced to achieve the purpose of saving bitrate. For example, there is a sequence of two frames to be encoded, the first frame is A and the second frame is B. Then in the encoding process, frame B will be encoded with frame A as a reference. Before formal encoding, TF will filter frame A with frame B at a filtering strength, that is, fuse part of the information in frame B into frame A. In subsequent encoding, frame B will be predicted and encoded with frame A as a reference. Since frame A already has some information of frame B at this time, the residual after prediction encoding will be reduced, enabling higher compression efficiency in subsequent transformation, quantization, and entropy coding, that is, saving bitstream. However, TF in current encoders has a problem of performance loss in some scenarios, resulting in the phenomenon that the bitstream does not decrease but increases. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a temporal filtering method, device, equipment and medium based on video block error, which can adjust the strength of temporal filtering according to the block error and block error distribution, and improve the ability of temporal filtering. The specific solutions are as follows:

[0005] In the first aspect, the present application discloses a temporal filtering method based on video block error, including:

[0006] Obtaining adjacent frames of the frame to be filtered and encoded as reference frames, and dividing the reference frames to obtain video blocks corresponding to each reference frame;

[0007] Determining the block error and block error distribution corresponding to the video blocks;

[0008] Determine a filtering weight corresponding to the video block based on the block error and the block error distribution;

[0009] Perform temporal filtering on the encoding frame to be filtered according to the filtering weight.

[0010] Optionally, determining the block error corresponding to the video block includes:

[0011] Determine a block error corresponding to the current video block based on the variance of the current video block, the sum of squared differences between the current video block and a matching block corresponding to the current video block, and a target parameter;

[0012] The matching block is determined by motion search; the target parameter includes a first parameter determined based on the bit depth.

[0013] Optionally, determining the block error distribution corresponding to the video block includes:

[0014] Determine the number of block pixels corresponding to the current video block according to the width and height of the current video block;

[0015] Generate a target value representing the difference between the width and height of the current video block;

[0016] Determine the block error distribution corresponding to the current video block based on the number of block pixels, the target value, and difference data between the current video block and a matching block corresponding to the current video block;

[0017] The difference data includes the sum of squared differences, the horizontal error sum, and the vertical error sum; the matching block is determined by motion search.

[0018] Optionally, obtaining adjacent frames of the encoding frame to be filtered as reference frames and partitioning the reference frames to obtain video blocks corresponding to the reference frames includes:

[0019] Obtain the previous N video frames and the subsequent N video frames adjacent to the encoding frame to be filtered to obtain reference frames corresponding to the encoding frame to be filtered;

[0020] Partition the reference frames into a plurality of initial blocks according to 16×16;

[0021] Perform motion estimation and motion compensation on the initial blocks, and obtain video blocks corresponding to the reference frames based on the compensated initial blocks.

[0022] Optionally, the determining the filtering weight corresponding to the video block based on the block error and the block error distribution includes:

[0023] Determine the base weight corresponding to the current video block according to the correspondence between the pre-built block error and the base weight;

[0024] Determine an adjustment coefficient according to the block error distribution, where the value of the block error distribution is inversely proportional to the adjustment coefficient; use the adjustment coefficient to adjust the base weight corresponding to the current video block, and obtain the filtering weight corresponding to the current video block according to the adjustment result.

[0025] Optionally, determining the filtering weight corresponding to the video block based on the block error and the block error distribution includes:

[0026] Compare the block error with a first error threshold;

[0027] If the block error is greater than the first error threshold, use a first preset weight as the base weight;

[0028] If the block error is not greater than the first error threshold, compare the block error with a second error threshold;

[0029] If the block error is greater than the second error threshold, use a second preset weight as the base weight; the first error threshold is greater than the second error threshold;

[0030] If the block error is not greater than the second error threshold, use a third preset weight as the base weight; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight;

[0031] Compare the block error distribution with a distribution threshold;

[0032] If the block error distribution is greater than the distribution threshold, obtain an adjusted weight based on the product of the base weight and a preset coefficient, and use the adjusted weight as the filtering weight of the video block;

[0033] If the block error distribution is not greater than the distribution threshold, directly use the base weight as the filtering weight of the video block.

[0034] Optionally, performing temporal filtering on the frame to be filtered and encoded according to the filtering weight includes:

[0035] Perform temporal filtering on the pixels at the target position in the frame to be filtered and encoded according to the filtering weights corresponding to the video blocks to which the pixels at the target position in each reference frame belong, and the reference pixel values corresponding to the pixels at the target position, to obtain filtered pixel values;

[0036] Obtain a filtered image frame corresponding to the frame to be filtered and encoded according to the filtered pixel values corresponding to the pixels at different positions in the frame to be filtered and encoded.

[0037] In a second aspect, the present application discloses a temporal filtering device based on video block errors, including:

[0038] A partitioning module, configured to obtain adjacent frames of a frame to be filtered and coded as reference frames, and partition the reference frames to obtain video blocks corresponding to the reference frames;

[0039] An error determination module, configured to determine the block error and block error distribution corresponding to the video block;

[0040] A filtering weight determination module, configured to determine the filtering weight corresponding to the video block based on the block error and the block error distribution;

[0041] A filtering module, configured to perform temporal filtering on the frame to be filtered and coded according to the filtering weight.

[0042] In a third aspect, the present application discloses an electronic device, including:

[0043] A memory, configured to store a computer program;

[0044] A processor, configured to execute the computer program to implement the foregoing temporal filtering method based on video block errors.

[0045] In a fourth aspect, the present application discloses a computer-readable storage medium, configured to store a computer program; wherein when the computer program is executed by a processor, the foregoing temporal filtering method based on video block errors is implemented.

[0046] In the present application, adjacent frames of a frame to be filtered and coded are obtained as reference frames, the reference frames are partitioned to obtain video blocks corresponding to the reference frames; the block error and block error distribution corresponding to the video block are determined; the filtering weight corresponding to the video block is determined based on the block error and the block error distribution; and temporal filtering is performed on the frame to be filtered and coded according to the filtering weight. It can be seen that the filtering weights corresponding to each video block are determined according to the block error and block error distribution of the video block, so as to improve the accuracy of calculating the filtering weight in units of video blocks, and temporal filtering is performed on the frame to be filtered and coded by comprehensively considering the filtering weights of the video blocks in adjacent frames. Therefore, the intensity of temporal filtering can be adjusted according to the block error and block error distribution, the ability of temporal filtering is improved, and the problem of performance loss of the filter is eliminated or reduced. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0048] Figure 1 Flow chart of a temporal filtering method based on video block error provided for this application;

[0049] Figure 2 Flow chart of a specific temporal filtering method based on video block error provided for this application;

[0050] Figure 3 Schematic structural diagram of a temporal filtering device based on video block error provided for this application;

[0051] Figure 4 Structural diagram of an electronic device provided for this application. Detailed implementation manners

[0052] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] In the prior art, the TF in the encoder may have a problem of performance loss in some scenarios, resulting in the phenomenon that the bitstream does not decrease but increases. To overcome the above technical problems, this application proposes a temporal filtering method based on video block error, which can adjust the intensity of temporal filtering according to the block error and the block error distribution, and improve the ability of temporal filtering.

[0054] An embodiment of this application discloses a temporal filtering method based on video block error. Refer to Figure 1 As shown, the method may include the following steps:

[0055] Step S11: Obtain adjacent frames of the frame to be filtered and encoded as reference frames, and divide the reference frames to obtain video blocks corresponding to each reference frame.

[0056] In this embodiment, video data in YUV (luminance, chrominance, Luminance, Chrominance) format is input into the encoder. For the frame to be filtered and encoded, its adjacent video frames are included in the reference frame buffer. If the forward or backward partial frames do not exist, these frames are ignored. Then the reference frames are divided to obtain video blocks.

[0057] Specifically, obtaining adjacent frames of the to-be-filtered coded frame as reference frames and partitioning the reference frames to obtain video blocks corresponding to each reference frame may include: obtaining the forward N video frames and the backward N video frames adjacent to the to-be-filtered coded frame to obtain the reference frames corresponding to the to-be-filtered coded frame; for example, incorporating 8 frames adjacent to the to-be-filtered coded frame (4 forward frames and 4 backward frames) into the reference frame buffer. Then, partitioning the reference frames into multiple initial blocks according to 16×16; and performing motion estimation and motion compensation on the initial blocks, and the compensated initial blocks are the video blocks corresponding to the reference frames, that is, dividing the current frame into 16x16 small blocks and performing motion estimation and motion compensation in units of blocks.

[0058] Step S12: Determine the block error and block error distribution corresponding to the video block.

[0059] After partitioning to obtain video blocks, calculate the block error (BLK_ERR) and block error distribution (BLK_FREQ) of each video block. The error situation between video blocks can be reflected through the block error and block error distribution.

[0060] In some embodiments, determining the block error corresponding to the video block includes: determining the block error corresponding to the current video block based on the variance of the current video block, the sum of squared differences between the current video block and the matching block corresponding to the current video block, and a target parameter; the matching block is determined through motion search; the target parameter includes a first parameter determined based on the bit depth. That is, specifically, the block error of the video block can be calculated according to the variance of the video block, the sum of squared differences (SSD, Sum of Squared Difference) between the video block and its matching block, and the parameter determined according to the bit depth.

[0061] Among them, the formula for the sum of squared differences is as follows:

[0062] ;

[0063] Among them, cur(i,j) represents the value of the pixel at coordinates (i,j) on the current block; ref(i,j) represents the value of the pixel at coordinates (i,j) on the matching block obtained through motion search.

[0064] The specific formula for the block error is as follows:

[0065] BLK_ERR = a1(SSD + 5) / (V + 5)+(SSD / 3200a2); where a1 is a fixed parameter (such as 0.022), and a2 is the first parameter determined based on the above bit depth.

[0066] a2 = 12 << (10 - internal_bit_depth), where internal_bit_depth represents the bit depth of the video frame where the current block is located, << is the left shift operator, and V is the variance of the video block.

[0067] In some embodiments, determining the block error distribution corresponding to the video block includes: determining the number of block pixels corresponding to the current video block according to the width and height of the current video block; generating a target value representing the difference between the width and height of the current video block; determining the block error distribution corresponding to the current video block based on the number of block pixels, the target value, and the difference data between the current video block and the matching block corresponding to the current video block; the difference data includes the sum of squared differences, the horizontal error sum, and the vertical error sum; the matching block is determined through motion search.

[0068] Among them, the calculation formula for the horizontal error sum (HSD, Horizontal Sum of Differences) is:

[0069] ;

[0070] The calculation formula for the vertical error sum (VSD, Vertical Sum of Differences) is:

[0071] ;

[0072] Among them, cur(i, j) represents the value of the pixel point with coordinates (i, j) on the current block; ref(i, j) represents the value of the pixel point with coordinates (i, j) on the matching block obtained through motion search.

[0073] The number of block pixels (cntV), , blk_size_x represents the width of the block, and blk_size_y represents the height of the block. The target value representing the difference between the width and height of the video block can specifically be: .

[0074] The calculation formula for the block error distribution can specifically be:

[0075] ; where b1 is a parameter.

[0076] Step S13: Determine the filtering weight corresponding to the video block based on the block error and the block error distribution.

[0077] According to the block error and the block error distribution of the current video block, the filtering weight corresponding to the current video block can be determined. By traversing the operations of the above Step S12 and Step S13 for all video blocks, the filtering weight corresponding to each video block can be obtained.

[0078] In some embodiments, determining the filtering weight corresponding to the video block based on the block error and the block error distribution may specifically include: determining the basic weight corresponding to the current video block according to the pre-established correspondence between the block error and the basic weight; that is, there is a correspondence between the block error and the basic weight, and the larger the block error, the smaller the basic weight. The above correspondence may specifically be in the form of a table or a function, and this embodiment does not make specific limitations. It can be seen that the larger the block error, the smaller the basic weight, and the smaller the basic weight, the smaller the filtering weight. The filtering weight represents the filtering intensity, that is, the lower the filtering intensity, so that when the error in the block is too large, the filtering intensity can be reduced.

[0079] Then, determine an adjustment coefficient according to the block error distribution, and the value of the block error distribution is inversely proportional to the adjustment coefficient; the value of the block error distribution is inversely proportional to the adjustment coefficient; use the adjustment coefficient to adjust the basic weight corresponding to the current video block, and obtain the filtering weight corresponding to the current video block according to the adjustment result; that is, after determining the basic weight according to the block error, then adjust the basic weight according to the block error distribution as appropriate to obtain the filtering weight corresponding to the video block for filtering. Of course, the basic weight can also be adjusted by using the original filtering weight to obtain the filtering weight. The original filtering weight refers to the filtering weight used by existing encoders in time-domain filtering in various coding standards. The larger and more dispersed the block error distribution is, through the adjustment coefficient, when the block error distribution is concentrated (that is, large errors appear in a small range), it is still possible to effectively filter most of the ranges with small errors.

[0080] In a specific embodiment, determining the filtering weight corresponding to the video block based on the block error and the block error distribution includes: comparing the block error with a first error threshold (BLK_THR1); if the block error is greater than the first error threshold, use a first preset weight as the basic weight (BASE_WEIGHT); if the block error is not greater than the first error threshold, compare the block error with a second error threshold (BLK_THR2); if the block error is greater than the second error threshold, use a second preset weight as the basic weight; the first error threshold is greater than the second error threshold; if the block error is not greater than the second error threshold, use a third preset weight as the basic weight; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight; compare the block error distribution with a distribution threshold (FREQ_THR); if the block error distribution is greater than the distribution threshold, obtain an adjusted weight based on the product of the basic weight and a preset coefficient, and use the adjusted weight as the filtering weight of the video block; the preset coefficient is less than 1; if the block error distribution is not greater than the distribution threshold, directly use the basic weight as the filtering weight of the video block.

[0081] Taking the first preset weight as 0.25, the second preset weight as 0.45, the third preset weight as 0.65, and the preset coefficient as 0.45 as an example for illustration, for example Figure 2 as shown below:

[0082] (1) Determine the values of BLK_THR and FREQ_THR; take BLK_THR1 = 100, BLK_THR2 = 50, FREQ_THR = 25;

[0083] (2) If BLK_ERR is greater than BLK_THR1, then set BASE_WEIGHT = 0.25 and go to step (5); otherwise go to step (3);

[0084] (3) If BLK_ERR is greater than BLK_THR2, then set BASE_WEIGHT = 0.45 and go to step (5); otherwise go to step (4);

[0085] (4) Set BASE_WEIGHT = 0.65 and go to step (5);

[0086] (5) If BLK_FREQ is greater than FREQ_THR, then multiply BASE_WEIGHT by 0.45; otherwise, directly use BASE_WEIGHT as the final filtering weight.

[0087] Step S14: Perform temporal filtering on the coding frame to be filtered according to the filtering weight.

[0088] Finally, perform temporal filtering on the coding frame to be filtered according to the calculated filtering weight, thereby adjusting the intensity of temporal filtering according to the block error and the block error distribution, improving the ability of temporal filtering, and eliminating or reducing the problem of performance loss of the filter.

[0089] In some embodiments, the performing temporal filtering on the coding frame to be filtered according to the filtering weight includes: performing temporal filtering on the pixel at the target position in the coding frame to be filtered according to the filtering weight corresponding to the video block to which the pixel at the target position in each reference frame belongs, and the reference pixel value corresponding to the pixel at the target position, to obtain the filtered pixel value; obtaining the filtered image frame corresponding to the coding frame to be filtered according to the filtered pixel values corresponding to the pixels at different positions in the coding frame to be filtered. That is, comprehensively considering the filtering weights of the video blocks where the pixels at position (i, j) in different reference frames are located, and the reference pixel values of the pixels at (i, j), perform filtering on the video block at position (i, j) in the coding frame to be filtered.

[0090] Specifically, if 16x16 partitioning is used for reference frame partitioning, since filtering is usually performed in 8x8 blocks, it is necessary to re-partition the 16x16 image into four 8x8 blocks and perform temporal filtering operations on each pixel in each block:

[0091] ;

[0092] where pel(i,j) represents the pixel value at position (i,j) in the 8x8 block of the filtered image; k represents the reference frame, , 2N is the number of obtained reference frames; is the filtering weight of the video block to which the pixel point (i,j) belongs; is the value of the pixel point referred to by the current pixel (i,j) in the reference frame k. Save the finally obtained filtered image frame to the position where the image to be filtered and encoded is located, and enter the formal encoding process.

[0093] As can be seen from the above, in this embodiment, adjacent frames of the frame to be filtered and encoded are obtained as reference frames, and the reference frames are partitioned to obtain video blocks corresponding to each reference frame; the block error and block error distribution corresponding to the video block are determined; the filtering weight corresponding to the video block is determined based on the block error and the block error distribution; and temporal filtering is performed on the frame to be filtered and encoded according to the filtering weight. It can be seen that the filtering weights corresponding to each video block are determined according to the block error and block error distribution of the video block, so as to improve the accuracy of filtering weight calculation in units of video blocks, and temporal filtering is performed on the frame to be filtered and encoded by comprehensively considering the filtering weights of video blocks in adjacent frames. Therefore, the intensity of temporal filtering can be adjusted according to the block error and block error distribution, the ability of temporal filtering can be improved, and the problem of performance loss of the filter can be eliminated or reduced.

[0094] Correspondingly, an embodiment of the present application also discloses a temporal filtering device based on video block error. Refer to Figure 3 shown. This device includes:

[0095] A partitioning module 11, configured to obtain adjacent frames of the frame to be filtered and encoded as reference frames, and partition the reference frames to obtain video blocks corresponding to each reference frame;

[0096] An error determination module 12, configured to determine the block error and block error distribution corresponding to the video block;

[0097] A filtering weight determination module 13, configured to determine the filtering weight corresponding to the video block based on the block error and the block error distribution;

[0098] A filtering module 14, configured to perform temporal filtering on the frame to be filtered and encoded according to the filtering weight.

[0099] As can be seen from the above, in this embodiment, the adjacent frames of the to-be-filtered coded frame are obtained as reference frames, and the reference frames are partitioned to obtain video blocks corresponding to the reference frames; the block error and the block error distribution corresponding to the video blocks are determined; the filtering weight corresponding to the video block is determined based on the block error and the block error distribution; and the to-be-filtered coded frame is subjected to temporal filtering according to the filtering weight. It can be seen that the filtering weights corresponding to the video blocks are determined according to the block error and the block error distribution of the video blocks, so as to improve the accuracy of the filtering weight calculation in units of video blocks, and the to-be-filtered coded frame is subjected to temporal filtering by integrating the filtering weights of the video blocks in the adjacent frames. Therefore, the intensity of the temporal filtering can be adjusted according to the block error and the block error distribution, the temporal filtering ability can be improved, and the problem of performance loss of the filter can be eliminated or reduced.

[0100] In some specific embodiments, the error determination module 12 may specifically include:

[0101] A block error determination unit, configured to determine a block error corresponding to a current video block based on the variance of the current video block, the sum of squared differences between the current video block and a matching block corresponding to the current video block, and a target parameter; the matching block is determined through motion search; the target parameter includes a first parameter determined based on the bit depth.

[0102] In some specific embodiments, the error determination module 12 may specifically include:

[0103] A block pixel number determination unit, configured to determine the number of block pixels corresponding to the current video block according to the width and height of the current video block;

[0104] A target value determination unit, configured to generate a target value representing the difference between the width and height of the current video block;

[0105] A block error distribution determination unit, configured to determine the block error distribution corresponding to the current video block based on the number of block pixels, the target value, and the difference data between the current video block and a matching block corresponding to the current video block; the difference data includes the sum of squared differences, the horizontal error sum, and the vertical error sum; the matching block is determined through motion search.

[0106] In some specific embodiments, the partitioning module 11 may specifically include:

[0107] A reference frame acquisition unit, configured to acquire the first N forward video frames and the last N backward video frames adjacent to the to-be-filtered coded frame to obtain the reference frames corresponding to the to-be-filtered coded frame;

[0108] A partitioning unit, configured to partition the reference frame into a plurality of initial blocks according to 16×16;

[0109] A motion compensation unit, configured to perform motion estimation and motion compensation on the initial block, and obtain a video block corresponding to the reference frame based on the compensated initial block.

[0110] In some specific embodiments, the filtering weight determination module 13 may specifically include:

[0111] A basic weight determination unit, configured to determine a basic weight corresponding to the current video block according to a pre-constructed correspondence between block error and basic weight;

[0112] A weight adjustment unit, configured to determine an adjustment coefficient according to the block error distribution, where the value of the block error distribution is inversely proportional to the adjustment coefficient; use the adjustment coefficient to adjust the basic weight corresponding to the current video block, and obtain the filtering weight corresponding to the current video block according to the adjustment result.

[0113] In some specific embodiments, the filtering weight determination module 13 may specifically include:

[0114] A first comparison unit, configured to compare the block error with a first error threshold;

[0115] A first basic weight determination unit, configured to use a first preset weight as the basic weight if the block error is greater than the first error threshold;

[0116] A second comparison unit, configured to compare the block error with a second error threshold if the block error is not greater than the first error threshold;

[0117] A second basic weight determination unit, configured to use a second preset weight as the basic weight if the block error is greater than the second error threshold; the first error threshold is greater than the second error threshold;

[0118] A third basic weight determination unit, configured to use a third preset weight as the basic weight if the block error is not greater than the second error threshold; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight;

[0119] A third comparison unit, configured to compare the block error distribution with a distribution threshold;

[0120] A first filtering weight determination unit, configured to obtain an adjusted weight based on the product of the basic weight and a preset coefficient if the block error distribution is greater than the distribution threshold, and use the adjusted weight as the filtering weight of the video block;

[0121] A second filtering weight determination unit, configured to directly use the basic weight as the filtering weight of the video block if the block error distribution is not greater than the distribution threshold.

[0122] In some specific embodiments, the filtering module 14 may specifically include:

[0123] A filtered pixel value calculation unit, configured to perform temporal filtering on the pixels at the target positions in the frame to be filtered and coded according to the filtering weights corresponding to the video blocks to which the pixels at the target positions in each reference frame respectively belong, and the reference pixel values corresponding to the pixels at the target positions, so as to obtain filtered pixel values;

[0124] A filtered image frame determination unit, configured to obtain a filtered image frame corresponding to the frame to be filtered and coded according to the filtered pixel values corresponding to the pixels at different positions in the frame to be filtered and coded.

[0125] Furthermore, an embodiment of the present application also discloses an electronic device. Refer to Figure 4 shown in the figure. The content in the figure shall not be considered as any limitation to the scope of use of the present application.

[0126] Figure 4 FIG. is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Wherein, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the temporal filtering method based on video block errors disclosed in any of the foregoing embodiments.

[0127] In this embodiment, the power supply 23 is used to provide operating voltages for the various hardware devices on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and specific limitations are not imposed thereon herein; the input / output interface 25 is used to obtain external input data or output data to the outside, and the specific interface type thereof can be selected according to specific application requirements, and no specific limitations are imposed herein.

[0128] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon include an operating system 221, a computer program 222, and data 223 including error distributions, etc., and the storage method may be short-term storage or permanent storage.

[0129] Among them, the operating system 221 is used to manage and control each hardware device and computer program 222 on the electronic device 20, so as to implement the operation and processing of the massive data 223 in the memory 22 by the processor 21. It can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the time-domain filtering method based on video block error executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0130] Furthermore, an embodiment of the present application also discloses a computer storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the steps of the time-domain filtering method based on video block error disclosed in any of the foregoing embodiments are implemented.

[0131] In this specification, the various embodiments are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0132] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, a software module executed by a processor, or a combination of both. The software module can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0133] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or device including the element.

[0134] The above has introduced in detail a temporal filtering method, device, equipment and medium based on video block error. In this article, specific examples are used to expound the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A temporal filtering method based on video block error, characterized in that Including: Obtain adjacent frames of the frame to be filtered and coded as reference frames, and divide the reference frames to obtain video blocks corresponding to the reference frames; Determine the block error and block error distribution corresponding to the video block; Determine the filtering weight corresponding to the video block based on the block error and the block error distribution; Perform temporal filtering on the frame to be filtered and coded according to the filtering weight; Wherein, determining the block error distribution corresponding to the video block includes: Determine the number of block pixels corresponding to the current video block according to the width and height of the current video block; Generate a target value representing the difference between the width and height of the current video block; Determine the block error distribution corresponding to the current video block based on the number of block pixels, the target value, and the difference data between the current video block and the matching block corresponding to the current video block; The difference data includes the sum of squared differences, horizontal error sum, and vertical error sum; the matching block is determined by motion search; Wherein, determining the filtering weight corresponding to the video block based on the block error and the block error distribution includes: Determine the basic weight corresponding to the current video block according to the pre-constructed correspondence between the block error and the basic weight; Determine an adjustment coefficient according to the block error distribution, and the value of the block error distribution is inversely proportional to the adjustment coefficient; use the adjustment coefficient to adjust the basic weight corresponding to the current video block, and obtain the filtering weight corresponding to the current video block according to the adjustment result.

2. The time-domain filtering method based on video block error according to claim 1, wherein Determine the block error corresponding to the video block, including: Determine the block error corresponding to the current video block based on the variance of the current video block, the sum of squared differences between the current video block and the matching block corresponding to the current video block, and the target parameter; The matching block is determined by motion search; the target parameter includes a first parameter determined based on the bit depth.

3. The time-domain filtering method based on video block error according to claim 1, wherein Obtain adjacent frames of the frame to be filtered and coded as reference frames, and divide the reference frames to obtain video blocks corresponding to the reference frames, including: Obtain the forward N video frames and the backward N video frames adjacent to the frame to be filtered and coded to obtain the reference frames corresponding to the frame to be filtered and coded; Divide the reference frames into multiple initial blocks according to 16×16; Perform motion estimation and motion compensation on the initial blocks, and obtain the video blocks corresponding to the reference frames based on the compensated initial blocks.

4. The time-domain filtering method based on video block error according to claim 1, characterized in that, Determine the filtering weight corresponding to the video block based on the block error and the block error distribution, including: Compare the block error with a first error threshold; If the block error is greater than the first error threshold, use a first preset weight as the basic weight; If the block error is not greater than the first error threshold, compare the block error with a second error threshold; If the block error is greater than the second error threshold, use a second preset weight as the basic weight; the first error threshold is greater than the second error threshold; If the block error is not greater than the second error threshold, use a third preset weight as the basic weight; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight; Compare the block error distribution with a distribution threshold; If the block error distribution is greater than the distribution threshold, an adjusted weight is obtained based on the product of the base weight and a preset coefficient, and the adjusted weight is used as the filtering weight of the video block; If the block error distribution is not greater than the distribution threshold, the base weight is directly used as the filtering weight of the video block.

5. The time-domain filtering method based on video block error according to any one of claims 1 to 4, characterized in that Performing temporal filtering on the coding frame to be filtered according to the filtering weight, including: Performing temporal filtering on the pixel at the target position in the coding frame to be filtered according to the filtering weight corresponding to the video block to which the pixel at the target position in each reference frame belongs, and the reference pixel value corresponding to the pixel at the target position, to obtain a filtered pixel value; According to the filtered pixel values corresponding to the pixels at different positions in the coding frame to be filtered, a filtered image frame corresponding to the coding frame to be filtered is obtained.

6. A time-domain filtering device based on video block error, characterized in that, Including: A partitioning module, configured to obtain adjacent frames of the coding frame to be filtered as reference frames, and partition the reference frames to obtain video blocks corresponding to the reference frames; An error determination module, configured to determine the block error and the block error distribution corresponding to the video block; A filtering weight determination module, configured to determine the filtering weight corresponding to the video block based on the block error and the block error distribution; A filtering module, configured to perform temporal filtering on the coding frame to be filtered according to the filtering weight; Among them, the error determination module is configured to determine the number of block pixels corresponding to the current video block according to the width and height of the current video block; generate a target value representing the difference between the width and height of the current video block; based on the number of block pixels, the target value, and the difference data between the current video block and the matching block corresponding to the current video block, determine the block error distribution corresponding to the current video block; the difference data includes the sum of squared differences, the horizontal error sum, and the vertical error sum; the matching block is determined by motion search; Among them, the filtering weight determination module is configured to determine the base weight corresponding to the current video block according to the pre-constructed correspondence between the block error and the base weight; determine an adjustment coefficient according to the block error distribution, and the value of the block error distribution is inversely proportional to the adjustment coefficient; adjust the base weight corresponding to the current video block by using the adjustment coefficient, and obtain the filtering weight corresponding to the current video block according to the adjustment result.

7. An electronic device, characterized in that, Including: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the temporal filtering method based on video block error according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, For storing a computer program; wherein when the computer program is executed by a processor, the temporal filtering method based on video block error according to any one of claims 1 to 5 is implemented.

Images