Time domain filtering method and device based on video block error, equipment and medium

Abstract

The invention discloses a time domain filtering method and device based on a video block error, equipment and a medium, and relates to the technical field of audio and video coding. The method comprises the following steps: determining a filtering weight corresponding to each video block according to block errors and block error distribution of the video blocks, improving the accuracy of filtering weight calculation by taking the video blocks as units, and performing time domain filtering on a to-be-filtered coding frame by synthesizing the filtering weights of the video blocks in adjacent frames, therefore, the time domain filtering intensity can be adjusted according to the block error and the block error distribution, the time domain filtering capability is improved, and the performance loss problem of the filter is eliminated or reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of audio - video coding, and particularly relates to a temporal filtering method, device, equipment and medium based on video block error. Background Technique

[0002] Audio - video coding standards are technical specifications for compressing and encoding audio and video data, aiming to reduce the amount of data for storage and transmission while ensuring a certain quality. Common audio - video coding standards include AVS3 (AudioVideo Standard3, the third - generation audio - video coding standard), H.265 / HEVC, H.266 / VVC, etc. An audio - video encoder is a software or hardware tool that implements the video compression algorithm of an audio - video coding standard, used to compress the original uncompressed video signal into a bitstream in a specific format for storage or transmission.

[0003] Temporal Filtering (TF for short) is a common tool in various encoders. By filtering an image that is more likely to be referenced in inter - frame coding in the temporal domain (note: whether a certain frame is likely to be referenced is specified artificially before coding), the residual of the frame encoded with this image as a reference is reduced, so as to achieve the purpose of saving bitrate. For example, there is a sequence of two frames to be encoded, the first frame is A and the second frame is B. Then in the encoding process, frame B will be encoded with frame A as a reference. Before formal encoding, TF will filter frame A with frame B at a filtering strength, that is, fuse part of the information in frame B into frame A. In subsequent encoding, frame B will perform predictive coding with frame A as a reference. Since frame A already has some information of frame B at this time, the residual after predictive coding will be reduced, enabling higher compression efficiency in subsequent transformation, quantization, and entropy coding, that is, saving bitstream. However, TF in current encoders has a problem of performance loss in some scenarios, resulting in the phenomenon that the bitstream does not decrease but increases. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a temporal filtering method, device, equipment and medium based on video block error, which can adjust the strength of temporal filtering according to block error and block error distribution, and improve the ability of temporal filtering. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a temporal filtering method based on video block error, including:

[0006] Obtain adjacent frames of the frame to be filtered and encoded as reference frames, and divide the reference frames to obtain video blocks corresponding to each reference frame;

[0007] Determine the block error and block error distribution corresponding to the video blocks;

[0008] Determine a filtering weight corresponding to the video block based on the block error and the block error distribution;

[0009] Perform temporal filtering on the encoding frame to be filtered according to the filtering weight.

[0010] Optionally, determining the block error corresponding to the video block includes:

[0011] Determine a block error corresponding to the current video block based on the variance of the current video block, the sum of squared differences between the current video block and a matching block corresponding to the current video block, and a target parameter;

[0012] The matching block is determined by motion search; the target parameter includes a first parameter determined based on the bit depth.

[0013] Optionally, determining the block error distribution corresponding to the video block includes:

[0014] Determine the number of block pixels corresponding to the current video block according to the width and height of the current video block;

[0015] Generate a target value representing the difference between the width and height of the current video block;

[0016] Determine the block error distribution corresponding to the current video block based on the number of block pixels, the target value, and the difference data between the current video block and a matching block corresponding to the current video block;

[0017] The difference data includes the sum of squared differences, the horizontal error sum, and the vertical error sum; the matching block is determined by motion search.

[0018] Optionally, obtaining adjacent frames of the encoding frame to be filtered as reference frames, and partitioning the reference frames to obtain video blocks corresponding to the reference frames includes:

[0019] Obtain the previous N video frames and the subsequent N video frames adjacent to the encoding frame to be filtered to obtain the reference frames corresponding to the encoding frame to be filtered;

[0020] Partition the reference frames into multiple initial blocks according to 16×16;

[0021] Perform motion estimation and motion compensation on the initial blocks, and obtain the video blocks corresponding to the reference frames based on the compensated initial blocks.

[0022] Optionally, the determining the filtering weight corresponding to the video block based on the block error and the block error distribution includes:

[0023] Determine the base weight corresponding to the current video block according to the correspondence between the pre-built block error and the base weight;

[0024] Determine an adjustment coefficient according to the block error distribution, where the value of the block error distribution is inversely proportional to the adjustment coefficient; adjust the base weight corresponding to the current video block using the adjustment coefficient, and obtain the filtering weight corresponding to the current video block according to the adjustment result.

[0025] Optionally, determining the filtering weight corresponding to the video block based on the block error and the block error distribution includes:

[0026] Compare the block error with a first error threshold;

[0027] If the block error is greater than the first error threshold, use a first preset weight as the base weight;

[0028] If the block error is not greater than the first error threshold, compare the block error with a second error threshold;

[0029] If the block error is greater than the second error threshold, use a second preset weight as the base weight; the first error threshold is greater than the second error threshold;

[0030] If the block error is not greater than the second error threshold, use a third preset weight as the base weight; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight;

[0031] Compare the block error distribution with a distribution threshold;

[0032] If the block error distribution is greater than the distribution threshold, obtain an adjusted weight based on the product of the base weight and a preset coefficient, and use the adjusted weight as the filtering weight of the video block;

[0033] If the block error distribution is not greater than the distribution threshold, directly use the base weight as the filtering weight of the video block.

[0034] Optionally, performing temporal filtering on the frame to be filtered and encoded according to the filtering weight includes:

[0035] Perform temporal filtering on the pixel at the target position in the frame to be filtered and encoded according to the filtering weight corresponding to the video block to which the pixel at the target position in each reference frame belongs, and the reference pixel value corresponding to the pixel at the target position, to obtain a filtered pixel value;

[0036] Obtain the filtered image frame corresponding to the frame to be filtered and encoded according to the filtered pixel values corresponding to the pixels at different positions in the frame to be filtered and encoded.

[0037] In a second aspect, the present application discloses a temporal filtering device based on video block error, including:

[0038] A partitioning module, configured to obtain adjacent frames of the frame to be filtered and encoded as reference frames, and partition the reference frames to obtain video blocks corresponding to the respective reference frames;

[0039] An error determination module, configured to determine the block error and the block error distribution corresponding to the video block;

[0040] A filtering weight determination module, configured to determine the filtering weight corresponding to the video block based on the block error and the block error distribution;

[0041] A filtering module, configured to perform temporal filtering on the frame to be filtered and encoded according to the filtering weight.

[0042] In a third aspect, the present application discloses an electronic device, including:

[0043] A memory, configured to store a computer program;

[0044] A processor, configured to execute the computer program to implement the foregoing temporal filtering method based on video block error.

[0045] In a fourth aspect, the present application discloses a computer-readable storage medium, configured to store a computer program; wherein when the computer program is executed by a processor, the foregoing temporal filtering method based on video block error is implemented.

[0046] In the present application, adjacent frames of the frame to be filtered and encoded are obtained as reference frames, the reference frames are partitioned to obtain video blocks corresponding to the respective reference frames; the block error and the block error distribution corresponding to the video block are determined; the filtering weight corresponding to the video block is determined based on the block error and the block error distribution; and temporal filtering is performed on the frame to be filtered and encoded according to the filtering weight. It can be seen that the filtering weight corresponding to each video block is determined according to the block error and the block error distribution of the video block, the accuracy of filtering weight calculation is improved in units of video blocks, and temporal filtering is performed on the frame to be filtered and encoded by comprehensively considering the filtering weights of the video blocks in adjacent frames. Therefore, the intensity of temporal filtering can be adjusted according to the block error and the block error distribution, the ability of temporal filtering is improved, and the problem of performance loss of the filter is eliminated or reduced. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0048] Figure 1 Flow chart of a temporal filtering method based on video block error provided for this application;

[0049] Figure 2 Flow chart of a specific temporal filtering method based on video block error provided for this application;

[0050] Figure 3 Schematic structural diagram of a temporal filtering device based on video block error provided for this application;

[0051] Figure 4 Structural diagram of an electronic device provided for this application. Detailed implementation manners

[0052] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] In the prior art, TF in an encoder may have a problem of performance loss in some scenarios, resulting in a phenomenon that the bitstream does not decrease but increases. To overcome the above technical problems, this application proposes a temporal filtering method based on video block error, which can adjust the intensity of temporal filtering according to the block error and block error distribution, and improve the ability of temporal filtering.

[0054] An embodiment of this application discloses a temporal filtering method based on video block error. Refer to Figure 1 As shown, the method may include the following steps:

[0055] Step S11: Obtain adjacent frames of the frame to be filtered and encoded as reference frames, and divide the reference frames to obtain video blocks corresponding to each reference frame.

[0056] In this embodiment, video data in YUV (luminance, chrominance, Luminance, Chrominance) format is input into the encoder. For the frame to be filtered and encoded, its adjacent video frames are incorporated into the reference frame buffer. If the forward or backward partial frames do not exist, these frames are ignored. Then the reference frames are divided to obtain video blocks.

[0057] Specifically, obtaining adjacent frames of the to-be-filtered coded frame as reference frames and partitioning the reference frames to obtain video blocks corresponding to each reference frame may include: obtaining the forward N video frames and the backward N video frames adjacent to the to-be-filtered coded frame to obtain the reference frames corresponding to the to-be-filtered coded frame; for example, incorporating 8 frames adjacent to the to-be-filtered coded frame (4 forward frames and 4 backward frames) into the reference frame buffer. Then, partitioning the reference frames into multiple initial blocks according to 16×16; and performing motion estimation and motion compensation on the initial blocks, and the compensated initial blocks are the video blocks corresponding to the reference frames, that is, dividing the current frame into 16x16 small blocks and performing motion estimation and motion compensation in units of blocks.

[0058] Step S12: Determine the block error and block error distribution corresponding to the video block.

[0059] After partitioning to obtain the video blocks, calculate the block error (BLK_ERR) and block error distribution (BLK_FREQ) of each video block. The error situation between video blocks can be reflected through the block error and block error distribution.

[0060] In some embodiments, determining the block error corresponding to the video block includes: determining the block error corresponding to the current video block based on the variance of the current video block, the sum of squared differences between the current video block and the matching block corresponding to the current video block, and a target parameter; the matching block is determined through motion search; the target parameter includes a first parameter determined based on the bit depth. That is, specifically, the block error of the video block can be calculated according to the variance of the video block, the sum of squared differences (SSD, Sum of Squared Difference) between the video block and its matching block, and the parameter determined according to the bit depth.

[0061] Among them, the formula for the sum of squared differences is as follows:

[0062] ;

[0063] Among them, cur(i,j) represents the value of the pixel point with coordinates (i,j) on the current block; ref(i,j) represents the value of the pixel point with coordinates (i,j) on the matching block obtained through motion search.

[0064] The specific formula for the block error is as follows:

[0065] BLK_ERR = a1(SSD + 5) / (V + 5)+(SSD / 3200a2); where a1 is a fixed parameter (such as 0.022), and a2 is the first parameter determined based on the above bit depth.

[0066] a2 = 12 << (10 - internal_bit_depth), where internal_bit_depth represents the bit depth of the video frame where the current block is located, << is the left shift operator, and V is the variance of the video block.

[0067] In some embodiments, determining the block error distribution corresponding to the video block includes: determining the number of block pixels corresponding to the current video block according to the width and height of the current video block; generating a target value representing the difference between the width and height of the current video block; determining the block error distribution corresponding to the current video block based on the number of block pixels, the target value, and the difference data between the current video block and the matching block corresponding to the current video block; the difference data includes the sum of squared differences, the horizontal error sum, and the vertical error sum; the matching block is determined by motion search.

[0068] Among them, the calculation formula for the horizontal error sum (HSD, Horizontal Sum of Differences) is:

[0069] ;

[0070] The calculation formula for the vertical error sum (VSD, Vertical Sum of Differences) is:

[0071] ;

[0072] Among them, cur(i, j) represents the value of the pixel at coordinates (i, j) on the current block; ref(i, j) represents the value of the pixel at coordinates (i, j) on the matching block obtained by motion search.

[0073] The number of block pixels (cntV), , blk_size_x represents the width of the block, and blk_size_y represents the height of the block. The target value representing the difference between the width and height of the video block can specifically be: .

[0074] The calculation formula for the block error distribution can specifically be:

[0075] ; where b1 is a parameter.

[0076] Step S13: Determine the filtering weight corresponding to the video block based on the block error and the block error distribution.

[0077] According to the block error and the block error distribution of the current video block, the filtering weight corresponding to the current video block can be determined. By traversing the operations of the above Step S12 and Step S13 for all video blocks, the filtering weight corresponding to each video block can be obtained.

[0078] In some embodiments, determining the filtering weight corresponding to the video block based on the block error and the block error distribution may specifically include: determining the basic weight corresponding to the current video block according to the pre-constructed correspondence between the block error and the basic weight; that is, there is a correspondence between the block error and the basic weight, and the larger the block error, the smaller the basic weight. The above correspondence may specifically be in the form of a table or a function, and this embodiment does not make specific limitations. It can be seen that the larger the block error, the smaller the basic weight, and the smaller the basic weight, the smaller the filtering weight. The filtering weight represents the filtering intensity, that is, the lower the filtering intensity, so that when the error in the block is too large, the filtering intensity can be reduced.

[0079] Then, determine the adjustment coefficient according to the block error distribution. The value of the block error distribution is inversely proportional to the adjustment coefficient; the value of the block error distribution is inversely proportional to the adjustment coefficient; use the adjustment coefficient to adjust the basic weight corresponding to the current video block, and obtain the filtering weight corresponding to the current video block according to the adjustment result; that is, after determining the basic weight according to the block error, then adjust the basic weight according to the block error distribution as appropriate to obtain the filtering weight for filtering corresponding to the video block. Of course, the basic weight can also be adjusted by using the original filtering weight to obtain the filtering weight. The original filtering weight refers to the filtering weight adopted by the existing encoder in time-domain filtering in various coding standards. The larger and more dispersed the block error distribution is, through the adjustment coefficient, when the block error distribution is concentrated (that is, large errors appear in a small range), it is still possible to effectively filter most of the range with small errors.

[0080] In a specific embodiment, determining the filtering weight corresponding to the video block based on the block error and the block error distribution includes: comparing the block error with a first error threshold (BLK_THR1); if the block error is greater than the first error threshold, use a first preset weight as the basic weight (BASE_WEIGHT); if the block error is not greater than the first error threshold, compare the block error with a second error threshold (BLK_THR2); if the block error is greater than the second error threshold, use a second preset weight as the basic weight; the first error threshold is greater than the second error threshold; if the block error is not greater than the second error threshold, use a third preset weight as the basic weight; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight; compare the block error distribution with a distribution threshold (FREQ_THR); if the block error distribution is greater than the distribution threshold, obtain an adjusted weight based on the product of the basic weight and a preset coefficient, and use the adjusted weight as the filtering weight of the video block; the preset coefficient is less than 1; if the block error distribution is not greater than the distribution threshold, directly use the basic weight as the filtering weight of the video block.

[0081] Taking the first preset weight as 0.25, the second preset weight as 0.45, the third preset weight as 0.65, and the preset coefficient as 0.45 as an example, the following is an illustration. For example Figure 2 as shown:

[0082] (1) Determine the values of BLK_THR and FREQ_THR; take BLK_THR1 = 100, BLK_THR2 = 50, and FREQ_THR = 25;

[0083] (2) If BLK_ERR is greater than BLK_THR1, then set BASE_WEIGHT = 0.25 and go to step (5); otherwise, go to step (3);

[0084] (3) If BLK_ERR is greater than BLK_THR2, then set BASE_WEIGHT = 0.45 and go to step (5); otherwise, go to step (4);

[0085] (4) Set BASE_WEIGHT = 0.65 and go to step (5);

[0086] (5) If BLK_FREQ is greater than FREQ_THR, then multiply BASE_WEIGHT by 0.45; otherwise, directly use BASE_WEIGHT as the final filtering weight.

[0087] Step S14: Perform temporal filtering on the coding frame to be filtered according to the filtering weight.

[0088] Finally, perform temporal filtering on the coding frame to be filtered according to the calculated filtering weight, thereby adjusting the intensity of temporal filtering according to the block error and the block error distribution, improving the ability of temporal filtering, and eliminating or reducing the problem of performance loss of the filter.

[0089] In some embodiments, the performing temporal filtering on the coding frame to be filtered according to the filtering weight includes: performing temporal filtering on the pixel at the target position in the coding frame to be filtered according to the filtering weight corresponding to the video block to which the pixel at the target position in each reference frame belongs, and the reference pixel value corresponding to the pixel at the target position, to obtain the filtered pixel value; obtaining the filtered image frame corresponding to the coding frame to be filtered according to the filtered pixel values corresponding to the pixels at different positions in the coding frame to be filtered. That is, comprehensively considering the filtering weights of the video blocks where the pixels at position (i, j) in different reference frames are located, and the reference pixel values of the pixels at (i, j), to filter the video block at position (i, j) in the coding frame to be filtered.

[0090] Specifically, if 16x16 partitioning is used during reference frame partitioning, since filtering is usually performed in 8x8 blocks, it is necessary to re-partition the 16x16 image into four 8x8 blocks and perform temporal filtering operations on each pixel in each block:

[0091] ;

[0092] where pel(i,j) represents the pixel value at position (i,j) in the 8x8 block of the filtered image; k represents the reference frame, , and 2N is the number of reference frames obtained; is the filtering weight of the video block to which the pixel point (i,j) belongs; is the value of the pixel point referred to by the current pixel (i,j) in the reference frame k. Save the finally obtained filtered image frame to the location where the image to be filtered and encoded is located, and enter the formal encoding process.

[0093] As can be seen from the above, in this embodiment, adjacent frames of the frame to be filtered and encoded are obtained as reference frames, and the reference frames are partitioned to obtain video blocks corresponding to each reference frame; the block error and block error distribution corresponding to the video block are determined; the filtering weight corresponding to the video block is determined based on the block error and the block error distribution; and temporal filtering is performed on the frame to be filtered and encoded according to the filtering weight. It can be seen that the filtering weights corresponding to each video block are determined according to the block error and block error distribution of the video block, so as to improve the accuracy of filtering weight calculation in units of video blocks, and temporal filtering is performed on the frame to be filtered and encoded by integrating the filtering weights of the video blocks in adjacent frames. Therefore, the intensity of temporal filtering can be adjusted according to the block error and block error distribution, the ability of temporal filtering can be improved, and the problem of performance loss of the filter can be eliminated or reduced.

[0094] Correspondingly, an embodiment of the present application also discloses a temporal filtering device based on video block error. Refer to Figure 3 as shown, this device includes:

[0095] A partitioning module 11, configured to obtain adjacent frames of the frame to be filtered and encoded as reference frames, and partition the reference frames to obtain video blocks corresponding to each reference frame;

[0096] An error determination module 12, configured to determine the block error and block error distribution corresponding to the video block;

[0097] A filtering weight determination module 13, configured to determine the filtering weight corresponding to the video block based on the block error and the block error distribution;

[0098] A filtering module 14, configured to perform temporal filtering on the frame to be filtered and encoded according to the filtering weight.

[0099] As can be seen above, in this embodiment, adjacent frames of the to-be-filtered coded frame are obtained as reference frames, and the reference frames are partitioned to obtain video blocks corresponding to the reference frames; the block error and block error distribution corresponding to the video blocks are determined; the filtering weight corresponding to the video blocks is determined based on the block error and the block error distribution; and the to-be-filtered coded frame is temporally filtered according to the filtering weight. It can be seen that the filtering weights corresponding to the video blocks are determined according to the block error and block error distribution of the video blocks, so as to improve the accuracy of filtering weight calculation in units of video blocks, and the to-be-filtered coded frame is temporally filtered by comprehensively considering the filtering weights of the video blocks in adjacent frames. Therefore, the intensity of temporal filtering can be adjusted according to the block error and block error distribution, the temporal filtering ability is improved, and the problem of performance loss of the filter is eliminated or reduced.

[0100] In some specific embodiments, the error determination module 12 may specifically include:

[0101] A block error determination unit, configured to determine a block error corresponding to a current video block based on the variance of the current video block, the sum of squared differences between the current video block and a matching block corresponding to the current video block, and a target parameter; the matching block is determined by motion search; the target parameter includes a first parameter determined based on the bit depth.

[0102] In some specific embodiments, the error determination module 12 may specifically include:

[0103] A block pixel number determination unit, configured to determine the number of block pixels corresponding to the current video block according to the width and height of the current video block;

[0104] A target value determination unit, configured to generate a target value representing the difference between the width and height of the current video block;

[0105] A block error distribution determination unit, configured to determine the block error distribution corresponding to the current video block based on the number of block pixels, the target value, and the difference data between the current video block and a matching block corresponding to the current video block; the difference data includes the sum of squared differences, the horizontal error sum, and the vertical error sum; the matching block is determined by motion search.

[0106] In some specific embodiments, the partitioning module 11 may specifically include:

[0107] A reference frame acquisition unit, configured to acquire the forward N video frames and the backward N video frames adjacent to the to-be-filtered coded frame to obtain the reference frames corresponding to the to-be-filtered coded frame;

[0108] A partitioning unit, configured to partition the reference frame into a plurality of initial blocks according to 16×16;

[0109] A motion compensation unit, configured to perform motion estimation and motion compensation on the initial block, and obtain a video block corresponding to the reference frame based on the compensated initial block.

[0110] In some specific embodiments, the filter weight determination module 13 may specifically include:

[0111] A basic weight determination unit, configured to determine a basic weight corresponding to the current video block according to a pre-established correspondence between block errors and basic weights;

[0112] A weight adjustment unit, configured to determine an adjustment coefficient according to the block error distribution, where the value of the block error distribution is inversely proportional to the adjustment coefficient; use the adjustment coefficient to adjust the basic weight corresponding to the current video block, and obtain the filter weight corresponding to the current video block according to the adjustment result.

[0113] In some specific embodiments, the filter weight determination module 13 may specifically include:

[0114] A first comparison unit, configured to compare the block error with a first error threshold;

[0115] A first basic weight determination unit, configured to use a first preset weight as the basic weight if the block error is greater than the first error threshold;

[0116] A second comparison unit, configured to compare the block error with a second error threshold if the block error is not greater than the first error threshold;

[0117] A second basic weight determination unit, configured to use a second preset weight as the basic weight if the block error is greater than the second error threshold; the first error threshold is greater than the second error threshold;

[0118] A third basic weight determination unit, configured to use a third preset weight as the basic weight if the block error is not greater than the second error threshold; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight;

[0119] A third comparison unit, configured to compare the block error distribution with a distribution threshold;

[0120] A first filter weight determination unit, configured to obtain an adjusted weight based on the product of the basic weight and a preset coefficient if the block error distribution is greater than the distribution threshold, and use the adjusted weight as the filter weight of the video block;

[0121] A second filter weight determination unit, configured to directly use the basic weight as the filter weight of the video block if the block error distribution is not greater than the distribution threshold.

[0122] In some specific embodiments, the filtering module 14 may specifically include:

[0123] A filtered pixel value calculation unit, configured to perform temporal filtering on the pixels at the target positions in the frame to be filtered and encoded according to the filtering weights corresponding to the video blocks to which the pixels at the target positions in each reference frame belong respectively, and the reference pixel values corresponding to the pixels at the target positions, so as to obtain filtered pixel values;

[0124] A filtered image frame determination unit, configured to obtain a filtered image frame corresponding to the frame to be filtered and encoded according to the filtered pixel values corresponding to the pixels at different positions in the frame to be filtered and encoded.

[0125] Furthermore, an embodiment of the present application also discloses an electronic device. Refer to Figure 4 as shown. The content in the figure shall not be construed as any limitation on the scope of use of the present application.

[0126] Figure 4 is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the temporal filtering method based on video block error disclosed in any of the foregoing embodiments.

[0127] In this embodiment, the power supply 23 is used to provide operating voltages for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed thereon here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application requirements, and no specific limitation is made here.

[0128] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon include an operating system 221, a computer program 222, and data 223 including error distributions, etc., and the storage method may be temporary storage or permanent storage.

[0129] Among them, the operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to implement the operation and processing of the massive data 223 in the memory 22 by the processor 21. It can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the time-domain filtering method based on video block error executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs that can be used to complete other specific tasks.

[0130] Furthermore, an embodiment of the present application also discloses a computer storage medium. Computer-executable instructions are stored in the computer storage medium. When the computer-executable instructions are loaded and executed by a processor, the steps of the time-domain filtering method based on video block error disclosed in any of the foregoing embodiments are implemented.

[0131] In this specification, the various embodiments are described in a progressive manner. The focus of each embodiment is on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description of the method part.

[0132] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0133] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or device including the element.

[0134] The above has introduced in detail a time-domain filtering method, device, equipment and medium based on video block error. In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A temporal filtering method based on video block error, characterized in that: include: Acquire adjacent frames of the frame to be filtered and encoded as reference frames, and divide the reference frames to obtain video blocks corresponding to the reference frames; Determining a block error and a block error distribution corresponding to the video block; Determining a filter weight corresponding to the video block based on the block error and the block error distribution; The to-be-filtered coded frame is subjected to time-domain filtering according to the filtering weight.

2. The temporal filtering method based on video block error according to claim 1, characterized in that: Determining a block error corresponding to the video block includes: Determine a block error corresponding to the current video block based on a variance of the current video block, a sum of squares of differences between the current video block and a matching block corresponding to the current video block, and a target parameter; The matching block is determined by motion search; and the target parameter includes a first parameter determined based on a bit depth.

3. The temporal filtering method based on video block error according to claim 1, characterized in that: Determining a block error distribution corresponding to the video block includes: Determining the number of block pixels corresponding to the current video block according to the width and height of the current video block; Generate a target value representing the difference between the width and height of the current video block; Determine a block error distribution corresponding to the current video block based on the number of block pixels, the target value, and difference data between the current video block and a matching block corresponding to the current video block; The difference data includes a difference square sum, a horizontal error sum, and a vertical error sum; and the matching block is determined by motion search.

4. The temporal filtering method based on video block error according to claim 1, characterized in that: Acquiring adjacent frames of the frame to be filtered and encoded as reference frames, and dividing the reference frames to obtain video blocks corresponding to the reference frames, including: Acquire N forward video frames and N backward video frames adjacent to the coded frame to be filtered, and obtain a reference frame corresponding to the coded frame to be filtered; Dividing the reference frame into 16×16 blocks to obtain a plurality of initial blocks; Motion estimation and motion compensation are performed on the initial block, and a video block corresponding to the reference frame is obtained based on the compensated initial block.

5. The temporal filtering method based on video block error according to claim 1, characterized in that: Determining a filtering weight corresponding to the video block based on the block error and the block error distribution includes: Determine the basic weight corresponding to the current video block according to the correspondence between the pre-constructed block error and the basic weight; An adjustment coefficient is determined according to the block error distribution, wherein the value of the block error distribution is inversely proportional to the adjustment coefficient; the basic weight corresponding to the current video block is adjusted using the adjustment coefficient, and a filter weight corresponding to the current video block is obtained according to the adjustment result.

6. The temporal filtering method based on video block error according to claim 5, characterized in that: Determining a filtering weight corresponding to the video block based on the block error and the block error distribution includes: comparing the block error to a first error threshold; If the block error is greater than the first error threshold, using the first preset weight as the basic weight; If the block error is not greater than the first error threshold, comparing the block error with a second error threshold; If the block error is greater than the second error threshold, the second preset weight is used as the basic weight; the first error threshold is greater than the second error threshold; If the block error is not greater than the second error threshold, the third preset weight is used as the basic weight; the first preset weight is less than the second preset weight, and the second preset weight is less than the third preset weight; comparing the block error distribution to a distribution threshold; If the block error distribution is greater than the distribution threshold, obtaining an adjusted weight based on the product of the basic weight and a preset coefficient, and using the adjusted weight as the filtering weight of the video block; If the block error distribution is not greater than the distribution threshold, the basic weight is directly used as the filtering weight of the video block.

7. The temporal filtering method based on video block error according to any one of claims 1 to 6, characterized in that: Performing time domain filtering on the to-be-filtered coded frame according to the filtering weights includes: According to the filtering weights corresponding to the video blocks to which the pixels at the target positions in each of the reference frames belong respectively, and the reference pixel values ​​corresponding to the pixels at the target positions, time-domain filtering is performed on the pixels at the target positions in the encoded frame to be filtered to obtain filtered pixel values; According to the filtered pixel values ​​corresponding to the pixels at different positions in the to-be-filtered coded frame, a filtered image frame corresponding to the to-be-filtered coded frame is obtained.

8. A temporal filtering device based on video block error, characterized in that: include: A division module, used for obtaining adjacent frames of the frame to be filtered and encoded as reference frames, and dividing the reference frames to obtain video blocks corresponding to each reference frame; An error determination module, configured to determine a block error and a block error distribution corresponding to the video block; A filter weight determination module, configured to determine a filter weight corresponding to the video block based on the block error and the block error distribution; A filtering module is used to perform time domain filtering on the encoded frame to be filtered according to the filtering weight.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the temporal filtering method based on video block error according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein when the computer program is executed by a processor, the time domain filtering method based on video block error according to any one of claims 1 to 7 is implemented.

Images