Time domain filtering method and device, electronic equipment and computer program product

Abstract

The invention provides a time domain filtering method and device, electronic equipment and a computer program product, and relates to the technical field of video coding and decoding. The method comprises the following steps: acquiring a video frame to be processed, wherein the video frame to be processed comprises a current image block; determining an adjacent block corresponding to the current image block, and determining a block filtering weight corresponding to the current image block according to the motion information of the adjacent block; based on a pre-configured filter, a pixel filtering weight corresponding to each pixel in the current image block is determined, and the block filtering weight and the pixel filtering weight are used for time domain filtering processing of the video. According to the invention, a scheme of weight adjustment based on motion information of adjacent blocks is introduced for a block level, and filtering processing is introduced for a pixel level to avoid the problem of overlarge unreasonable weight amplitude difference.

Description

Technical Field

[0001] The present disclosure relates to the field of video coding and decoding technology, and in particular to a time domain filtering method, a time domain filtering device, an electronic device and a computer program product. Background Art

[0002] Motion Compensated Temporal Filter (MCTF) is a pre-coding filtering algorithm widely used by many coding standards, such as High Efficiency Video Coding (HEVC) standard, Versatile Video Coding (VVC) standard, Audio Video coding Standard (AVS), etc. This algorithm is used to remove temporal noise of the video before encoding, thereby improving the overall coding compression rate of the video.

[0003] Generally speaking, MCTF will take effect at the lower levels in a Group of Pictures (GOP). For example, in GOP8, only frames at layers 0 and 1 will be filtered by MCTF. Because these frames are more important in the temporal reference structure, MCTF can be mainly decomposed into two modules, namely the Motion Compensation (MC) module and the Temporal Filtering (TF) module. The MV module is used to divide the image frame into multiple image blocks of specified sizes and search for corresponding matching blocks in the previous and next reference frames; the TF module uses the searched image blocks and the current block for weighted filtering. Summary of the invention

[0004] The present disclosure provides a time domain filtering method, device, electronic device, computer readable storage medium and computer program product to at least solve the problem that the time domain filtering scheme of video in the related art has a poor filtering effect due to the large difference in amplitude between the weights at the image block level and the pixel level. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a time domain filtering method is provided, comprising: obtaining a video frame to be processed, the video frame to be processed comprising a current image block; determining an adjacent block corresponding to the current image block, and determining a block filtering weight corresponding to the current image block according to motion information of the adjacent block; and determining a pixel filtering weight corresponding to each pixel in the current image block based on a pre-configured filter, wherein the block filtering weight and the pixel filtering weight are used for time domain filtering processing of the video.

[0006] In an exemplary embodiment of the present disclosure, the determining the neighboring blocks corresponding to the current image block and determining the block filtering weight corresponding to the current image block according to the motion information of the neighboring blocks include: obtaining a non-boundary image block in the video frame to be processed as the current image block; obtaining a plurality of image blocks spatially adjacent to the current image block as the neighboring blocks; determining block motion difference information between the neighboring blocks and the current image block according to the motion information of the plurality of neighboring blocks; and determining the block filtering weight based on the block motion difference information.

[0007] In an exemplary embodiment of the present disclosure, determining the block motion difference information between the adjacent blocks and the current image block according to the motion information of the plurality of adjacent blocks includes: determining a first motion vector corresponding to the current image block, the first motion vector including a first horizontal component and a first vertical component; determining a second motion vector corresponding to each of the plurality of adjacent blocks, the second motion vector including a second horizontal component and a second vertical component; and determining the block motion difference information according to the first horizontal component, the first vertical component, a plurality of the second horizontal components, and a plurality of the second vertical components.

[0008] In an exemplary embodiment of the present disclosure, determining the block filtering weight based on the block motion difference information includes: determining an initial block filtering weight corresponding to the current image block based on the block motion difference information; obtaining a frame type of the video frame to be processed, and determining a weight adjustment coefficient according to the frame type; and determining the block filtering weight according to the weight adjustment coefficient and the initial block filtering weight.

[0009] In an exemplary embodiment of the present disclosure, determining the initial block filtering weight corresponding to the current image block based on the block motion difference information includes: obtaining a preconfigured weight limiting function, the weight limiting function including a first weight value and a second weight value; determining a block weight adjustment condition corresponding to the current image block based on the block motion difference information; and determining the initial block filtering weight according to the first weight value, the second weight value and the block weight adjustment condition.

[0010] In an exemplary embodiment of the present disclosure, determining the pixel filtering weight corresponding to each pixel in the current image block based on a preconfigured filter includes: obtaining a preconfigured pixel filter, and determining the sum of filtering weights corresponding to the pixel filter; determining an initial pixel filtering weight corresponding to each pixel based on the pixel filter and the sum of filtering weights; and determining the pixel filtering weight of each pixel in the block based on the initial pixel filtering weight and the block filtering weight.

[0011] In an exemplary embodiment of the present disclosure, the method further includes: acquiring a boundary image block in the video frame to be processed; and configuring a block filtering weight corresponding to the boundary image block to a preset weight value.

[0012] According to a second aspect of an embodiment of the present disclosure, a time domain filtering device is provided, comprising: an image block determination module, used to obtain a video frame to be processed, the video frame to be processed includes a current image block; a block weight determination module, used to determine an adjacent block corresponding to the current image block, and determine a block filtering weight corresponding to the current image block according to motion information of the adjacent block; a pixel weight determination module, used to determine a pixel filtering weight corresponding to each pixel in the current image block based on a pre-configured filter, the block filtering weight and the pixel filtering weight are used for time domain filtering processing of the video.

[0013] In an exemplary embodiment of the present disclosure, the block weight determination module includes a block weight determination unit, which is used to: obtain a non-boundary image block in the video frame to be processed as the current image block; obtain a plurality of image blocks that are spatially adjacent to the current image block as the adjacent blocks; determine block motion difference information between the adjacent blocks and the current image block based on motion information of the plurality of adjacent blocks; and determine the block filtering weight based on the block motion difference information.

[0014] In an exemplary embodiment of the present disclosure, the block weight determination unit includes a difference information determination subunit, which is used to: determine a first motion vector corresponding to the current image block, the first motion vector including a first horizontal component and a first vertical component; determine a second motion vector corresponding to each of the plurality of adjacent blocks, the second motion vector including a second horizontal component and a second vertical component; determine the block motion difference information based on the first horizontal component, the first vertical component, a plurality of the second horizontal components, and a plurality of the second vertical components.

[0015] In an exemplary embodiment of the present disclosure, the block weight determination unit includes a block weight determination subunit, which is used to: determine the initial block filter weight corresponding to the current image block based on the block motion difference information; obtain the frame type of the video frame to be processed, and determine the weight adjustment coefficient according to the frame type; determine the block filter weight according to the weight adjustment coefficient and the initial block filter weight.

[0016] In an exemplary embodiment of the present disclosure, the block weight determination subunit includes an initial block weight determination subunit, which is used to: obtain a pre-configured weight limiting function, the weight limiting function including a first weight value and a second weight value; determine the block weight adjustment condition corresponding to the current image block based on the block motion difference information; determine the initial block filtering weight according to the first weight value, the second weight value and the block weight adjustment condition.

[0017] In an exemplary embodiment of the present disclosure, the pixel weight determination module includes a pixel weight determination unit, which is used to: obtain a preconfigured pixel filter and determine the filter weight sum corresponding to the pixel filter; determine the initial pixel filter weight corresponding to each of the pixels based on the pixel filter and the filter weight sum; determine the pixel filter weight of each pixel in the block based on the initial pixel filter weight and the block filter weight.

[0018] In an exemplary embodiment of the present disclosure, the above-mentioned time domain filtering device also includes a block weight configuration module, which is used to: obtain the boundary image block in the video frame to be processed; and configure the block filtering weight corresponding to the boundary image block to a preset weight value.

[0019] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any one of the time domain filtering methods described above.

[0020] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the time domain filtering methods described above.

[0021] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, which implements any one of the time domain filtering methods described above when executed by a processor.

[0022] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0023] On the one hand, by introducing the motion information of neighboring blocks to adjust the block-level filter weights of each image block in the processed video frame, the problem of large differences in block weights in time domain filtering can be avoided. On the other hand, based on the pre-configured filter, the pixel-level filter weights corresponding to each pixel in the block can be optimized to avoid the problem of unreasonable large differences in weight amplitudes that affect the filtering effect.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0026] Figure 1 This is a schematic diagram of the effectiveness of MCTF under GOP8.

[0027] Figure 2 This is a schematic diagram of the principle and implementation of MCTF.

[0028] Figure 3 It is a schematic diagram of the pixel values ​​of a 128x128 image block.

[0029] Figure 4 It is a schematic diagram of the filter weights for each pixel in a 128x128 image block.

[0030] Figure 5 The figure is a flow chart of a time domain filtering method according to an exemplary embodiment.

[0031] Figure 6 It is a block diagram of a time domain filtering device according to an exemplary embodiment.

[0032] Figure 7 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0033] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0035] MCTF is usually only effective at the lower level in a GOP, see Figure 1 As shown, Figure 1 This is a schematic diagram of MCTF taking effect under GOP8. In GOP8, only frames located at layer 0 and layer 1 will be filtered by MCTF. MCTF mainly includes motion compensation module and temporal filtering module. Figure 2 , Figure 2 This is the principle and implementation diagram of MCTF. For the time domain filtering module, I t Each 8x8 image block in the image is compared with 8 reference blocks searched in 8 adjacent reference frames. The reference frames are generally the four adjacent frames before and after (denoted as I t-4 ,I t-3 ,I t-2 ,I t-1 ,I t+1 ,I t+2 ,I t+3 ,I t+4 ), a total of 9 image blocks are weighted in a certain proportion (each block has a corresponding weight w, it should be noted here that w is an 8x8 weight block, which corresponds to the image block, and each pixel is multiplied by the weight value of the corresponding position). The result of time domain filtering is obtained, that is, a filtered 8x8 image block, which will replace I t The original image block in .

[0036] However, in the MCTF filtering algorithm, the weights between adjacent pixels vary greatly, and the overall filter weights of adjacent 8x8 blocks also vary greatly. Figure 3 , Figure 3 This is a schematic diagram of the pixel values ​​of a 128x128 image block. A 128x128 image block contains a total of 16*16=256 8x8 image blocks. Figure 4 , Figure 4 This is a schematic diagram of the filter weights for each pixel in a 128x128 image block. Figure 4 It can be seen that the difference between pixel weights is very large, which leads to large differences between pixels after filtering, affecting the filtering effect. Since MCTF does not fully consider the correlation between blocks during filtering, the content between adjacent 8x8 blocks in the filtered image is quite different and discontinuous, which further has a significant negative impact on the filtering effect.

[0037] Based on this, according to the embodiments of the present disclosure, a time domain filtering method, a time domain filtering device, an electronic device, a computer-readable storage medium, and a computer program product are proposed.

[0038] Figure 5 is a flow chart of a time domain filtering method according to an exemplary embodiment. Figure 5As shown, the time domain filtering method can be used in a computer device, wherein the computer device described in the present disclosure may include mobile terminal devices such as mobile phones, tablet computers, laptop computers, PDAs, and fixed terminal devices such as desktop computers. This exemplary embodiment uses the method applied to a computer device as an example, and it can be understood that the method can also be applied to a server, and can also be applied to a system including a computer device and a server, and is implemented through the interaction between the computer device and the server. Specifically, the following steps are included.

[0039] Step S510: obtaining a video frame to be processed, where the video frame to be processed includes a current image block.

[0040] Step S520, determining the adjacent blocks corresponding to the current image block, and determining the block filtering weight corresponding to the current image block according to the motion information of the adjacent blocks.

[0041] Step S530: Based on the pre-configured filter, determine the pixel filter weight corresponding to each pixel in the current image block, and the block filter weight and the pixel filter weight are used for time domain filtering processing of the video.

[0042] According to the temporal filtering method in this exemplary embodiment, on the one hand, by introducing the motion information of the neighboring blocks to adjust the block-level filtering weight of each image block in the processed video frame, the problem of excessive differences in block weights in temporal filtering can be avoided. On the other hand, based on the pre-configured filter, the pixel-level filtering weight corresponding to each pixel in the block can be optimized, which can avoid the problem of unreasonable large differences in weight amplitudes that affect the filtering effect.

[0043] The time domain filtering method in this exemplary embodiment will be further described below.

[0044] In step S510, a video frame to be processed is obtained, where the video frame to be processed includes a current image block.

[0045] In an exemplary embodiment of the present disclosure, the video frame to be processed may be a video frame to be processed in video encoding and decoding. The current image block may be an image block to be subjected to time domain filtering in the video frame to be processed. The image frame to be processed includes multiple image blocks. In the MCTF processing process, the image blocks included in the image frame to be processed are subjected to time domain filtering.

[0046] Before video encoding, a filtering algorithm can usually be used to remove temporal noise in the video, such as using the MCTF algorithm to perform temporal filtering on the video. The video to be processed consists of a sequence of video frames to be processed, and the video frames in the sequence of video frames to be processed can be processed frame by frame. For the currently acquired video frame to be processed, the video frame to be processed can contain multiple image blocks. For example, the video frame to be processed can be divided into multiple 8x8 image blocks, and one image block is selected from the multiple image blocks as the current image block. In other exemplary embodiments of the present disclosure, the video frame to be processed can also be divided into image blocks of other sizes according to specific needs.

[0047] In step S520, the adjacent blocks corresponding to the current image block are determined, and the block filtering weight corresponding to the current image block is determined according to the motion information of the adjacent blocks.

[0048] In an exemplary embodiment of the present disclosure, the adjacent block may be other image blocks adjacent to the current image block in the image frame to be processed. The motion information of the adjacent block may be motion vector information corresponding to the adjacent block. The block filtering weight may be the weight corresponding to the current image block in the time domain filtering process.

[0049] After the current image block is acquired, multiple image blocks adjacent to the current image block can be determined as adjacent blocks. Then, motion information corresponding to each of the multiple adjacent blocks is acquired, and then the block filter weight corresponding to the current image block is determined according to the motion information of the adjacent blocks.

[0050] In an exemplary embodiment of the present disclosure, for step S520, determining the adjacent blocks corresponding to the current image block, and determining the block filtering weight corresponding to the current image block according to the motion information of the adjacent blocks, includes: obtaining a non-boundary image block in the video frame to be processed as the current image block; obtaining multiple image blocks that are spatially adjacent to the current image block as adjacent blocks; determining block motion difference information between the adjacent blocks and the current image block according to the motion information of the multiple adjacent blocks; and determining the block filtering weight based on the block motion difference information.

[0051] The non-boundary image block may be an image block that is not located in a boundary region in the image frame to be processed. The block motion difference information may be difference information of motion information between the current image block and a plurality of adjacent blocks.

[0052] In this embodiment, the filter weight value of the non-boundary image block in the video frame to be processed can be adjusted first. One of the non-boundary image blocks in the video frame to be processed is selected as the current image block (such as an 8x8 image block), and the block filter weight of the current image block as a whole is determined. For the current image block, the four adjacent 8x8 blocks above, below, left and right of the block can be obtained as adjacent blocks; then the motion information of the above multiple adjacent blocks is obtained, that is, the motion vector (Motion Vector, MV) of each adjacent block, which are recorded as MV1, MV2, MV3, and MV4 respectively.

[0053] According to the MVs of multiple adjacent blocks, the block motion difference information between the adjacent blocks and the current image block is calculated, and then the block filter weight corresponding to the current image block is calculated according to the block motion difference information. By introducing the motion information of the adjacent blocks, the block filter weight used in the current image block can be optimized and adjusted to avoid large differences and discontinuities between image blocks.

[0054] In an exemplary embodiment of the present disclosure, determining block motion difference information between a neighboring block and a current image block according to motion information of a plurality of neighboring blocks includes: determining a first motion vector corresponding to the current image block, the first motion vector including a first horizontal component and a first vertical component; determining a second motion vector corresponding to each of the plurality of neighboring blocks, the second motion vector including a second horizontal component and a second vertical component; and determining the block motion difference information according to the first horizontal component, the first vertical component, a plurality of second horizontal components, and a plurality of second vertical components.

[0055] The first motion vector may be a motion vector corresponding to the current image block. The first horizontal component may be a horizontal component of the motion vector corresponding to the current image block. The first vertical component may be a vertical component of the motion vector corresponding to the current image block. The second motion vector may be a motion vector corresponding to an adjacent block. The second horizontal component may be a horizontal component of the motion vector corresponding to the adjacent block. The second vertical component may be a vertical component of the motion vector corresponding to the adjacent block.

[0056] Determine the first motion vector corresponding to the current image block. The motion vector is usually composed of a horizontal component and a vertical component. The horizontal component of the first motion vector is taken as the first horizontal component, which is recorded as The vertical component of the first motion vector is taken as the first vertical component, which is recorded as Similarly, the second motion vector corresponding to the adjacent block is determined, and the horizontal component of the second motion vector is used as the second horizontal component. Since there are multiple adjacent blocks, Identify the second horizontal component to distinguish multiple different adjacent blocks; take the vertical component of each second motion vector as the second vertical component, recorded as After the above motion information is determined, the block motion difference information may be calculated using Formula 1.

[0057]

[0058] Among them, V can represent the block motion difference information; MV i It can represent the MVs corresponding to the four adjacent blocks adjacent to the current image block; can represent the first horizontal component; The first vertical component can be represented; can represent the second horizontal component; The second vertical component may be represented. The block motion difference information between the current image block and the neighboring block may be calculated according to the motion information of the neighboring block.

[0059] In an exemplary embodiment of the present disclosure, block filter weights are determined based on block motion difference information, including: determining an initial block filter weight corresponding to a current image block based on the block motion difference information; obtaining a frame type of a video frame to be processed, and determining a weight adjustment coefficient according to the frame type; and determining a block filter weight according to the weight adjustment coefficient and the initial block filter weight.

[0060] The initial block filter weight may be a block-level filter weight corresponding to the current image block determined according to the block motion difference information. The frame type may be a frame type corresponding to the image frame to be processed, and the frame type may include an intra-coded frame (Intra-coded picture, I frame, often called a key frame), a predictive-coded picture frame (Predictive-coded picture, P frame) and a bi-directionally predicted picture frame (Bi-directionally predicted picture, B frame). The weight adjustment coefficient may be an adjustment coefficient determined according to the frame type, and is used to perform weighted adjustment on the value of the block filter weight.

[0061] After calculating the block motion difference information between the current image block and the adjacent blocks, the initial block filter weight corresponding to the current image block can be determined according to the block motion difference information. In an exemplary embodiment of the present disclosure, the initial block filter weight can be determined by the following steps: obtaining a pre-configured weight clipping function, the weight clipping function includes a first weight value and a second weight value; determining a block weight adjustment condition corresponding to the current image block based on the block motion difference information; determining the initial block filter weight according to the first weight value, the second weight value and the block weight adjustment condition.

[0062] The weight clipping function may be used to limit the initial block filter weight within a specific range. The first weight value may be a first weight boundary value configured in the weight clipping function. The second weight value may be a second weight boundary value configured in the weight clipping function.

[0063] For the calculation of block filter weights, a weight clipping function may be pre-configured, and the specific numerical value of the block filter weight may be limited to a specified range through the weight clipping function. For example, in this embodiment, the first weight value may be set to 0.5, and the second weight value may be set to 2.5. In addition, the block weight adjustment condition for weight value adjustment may be determined based on the calculated block motion difference information, and the initial block filter weight of the current image block may be determined in combination with the weight clipping function, as shown in Formula 2.

[0064] weight0=CLIP3(0.5,2.5,1+0.5*(1-V / 64)) (Formula 2)

[0065] Among them, weight0 can represent the initial block filter weight; 0.5*(1-V / 64) can be the block weight adjustment condition determined according to the block motion difference information V; CLIP3() can represent the weight limiting function; the first parameter in CLIP3() represents the first weight value, and the second parameter in CLIP3() represents the second weight value.

[0066] Formula 2 means: if the calculation result of the "0.5*(1-V / 64)" part is less than 0.5, the initial block filter weight weight0 is equal to 0.5; if the calculation result of the "0.5*(1-V / 64)" part is greater than 2.5, the initial block filter weight weight0 is equal to 2.5; otherwise, the initial block filter weight weight0 is equal to the calculation result of the "0.5*(1-V / 64)" part. The weight limit function can limit the block filter weight of the current image block to a specific range to avoid the problem of large differences in content between image blocks.

[0067] After calculating the initial block filter weight, the initial block filter weight can be further adjusted in combination with the frame type to obtain the final block filter weight. The specific adjustment strategy is: configure a corresponding weight adjustment coefficient for different frame types, and then calculate the block filter weight according to the weight adjustment coefficient. Specifically, as shown in Formula 3:

[0068] weight1=weight0*k (Formula 3)

[0069] Among them, weight1 can represent the block filter weight; weight0 can represent the initial block filter weight; k can represent the weight adjustment coefficient. The specific value of the weight adjustment coefficient is determined according to the frame type. k is a real number greater than. K of different frame types can take different values ​​or the same value. For example, for I frames, P frames and B frames, k takes values ​​of 0.75, 0.90 and 1.0 respectively.

[0070] Since MCTF may take effect at a lower level in a GOP, the effect of MCTF on different frame types may be different. Therefore, this embodiment adjusts the block filter weight of the current image block in combination with the frame type, so that the adjustment degree of different frame types is more consistent with the time domain filtering process.

[0071] In step S530, based on the pre-configured filter, a pixel filtering weight corresponding to each pixel in the current image block is determined.

[0072] In an exemplary embodiment of the present disclosure, the pixel filtering weight may be a weight corresponding to each pixel in the current image block during the time domain filtering process.

[0073] For the pixel-level filtering weight used for each pixel in the current image block, the present disclosure introduces a Gaussian filtering method to further optimize the pixel-level filtering weight. Specifically, for each pixel in the current image block, a filter can be pre-configured, and the filter is used to adjust the filtering weight of each pixel in the image block to obtain a pixel filtering weight.

[0074] In an exemplary embodiment of the present disclosure, for step S530, based on a preconfigured filter, the pixel filtering weight corresponding to each pixel in the current image block is determined, including: obtaining a preconfigured pixel filter, determining the sum of filtering weights corresponding to the pixel filter; determining the initial pixel filtering weight corresponding to each pixel based on the pixel filter and the sum of filtering weights; determining the pixel filtering weight of each pixel in the block based on the initial pixel filtering weight and the block filtering weight.

[0075] The pixel filter may be a pre-configured Gaussian filter for filtering and denoising each pixel value in the current image block. The filter weight sum may be the sum of the Gaussian weights of the Gaussian filter. The initial pixel filter weight may be a filter weight of an image pixel directly calculated based on the Gaussian filter.

[0076] For the filter weight w(x, y) of each pixel in the image block, this embodiment proposes to use a pixel filter (such as a Gaussian filter) of size SxS to adjust the weight, as shown in Formula 4.

[0077]

[0078] Among them, w gau(x,y) can represent the Gaussian filter weight of each pixel in the current image block; w(x,y) can represent the filter weight of the initial state of each pixel in the current image block; (x,y) can represent the pixel coordinates in the current image block, and the same applies to (i,j) and (x+i,y+j). gau(i,j) can represent the Gaussian weight at position (i,j). S can be any natural number greater than 1, and there is no requirement for the Gaussian weight. You can choose to derive any Gaussian weight, and sumGau is the sum of SxS Gaussian weights. For example, S=5, and a 5x5 Gaussian filter is used for weight adjustment. The pixel filter (i.e., the Gaussian filter weight gau) is specifically shown in Formula 5.

[0079]

[0080] Among them, gau can represent the pre-configured Gaussian filter weight; sumGau can represent the sum of the filter weights, such as sumGau = 48. Finally, the filter weight w of each pixel is filter (x, y) is determined by the block filter weight weight1 of formula (3) and the Gaussian filter weight of formula (4), as shown in formula 6.

[0081] w filter (x,y)=w gau (x,y)*weight1 (Formula 6)

[0082] Among them, w filter (x, y) can represent the pixel filter weight corresponding to each pixel; weight1 can represent the block filter weight; w gau (x, y) can represent the Gaussian filter weight of each pixel in the current image block. The above steps introduce Gaussian filtering to calculate the pixel-level filter weight to avoid the problem of unreasonable large differences in weight amplitudes between pixels.

[0083] In an exemplary embodiment of the present disclosure, the method further includes: acquiring a boundary image block in the video frame to be processed; and configuring a block filter weight corresponding to the boundary image block to a preset weight value.

[0084] The boundary image block may be an image block located in a boundary area of ​​the image frame to be processed. The preset weight value may be a pre-configured block weight value.

[0085] The present disclosure can also propose a weight determination scheme for boundary image blocks at the boundary of image blocks. For example, the block filter weights of the boundary image blocks at the boundary of the image can be configured as preset weight values, such as the block filter weights of the boundary image blocks are all equal to 1 by default. By configuring different block filter weights for boundary image blocks and non-boundary image blocks, the temporal filtering effect of the video can be further optimized.

[0086] After determining the block filter weights and pixel filter weights corresponding to the video frame to be processed, the above weights can be used in the video time domain filtering process. -Delta rate), which is used to evaluate the RD (rate-Rate, distortion-Distortion) performance indicators of different video encoders, and found that the present disclosure can obtain a BDSSIM611 gain of about 0.6% by weight optimization, wherein SSIM in BDSSIM611 can represent structural similarity (SSIM), 611 can represent the color coding model YUV components with a ratio of 6:1:1, Y represents brightness, U represents the first chrominance component, and V represents the second chrominance component; BDPSNR611 gain of about 0.3%, wherein PSNR can represent peak signal-to-noise ratio (Peak signal-to-noise ratio, often abbreviated as PSNR).

[0087] In summary, the time domain filtering method disclosed in the present invention obtains a video frame to be processed, and the video frame to be processed includes a current image block; determines the adjacent blocks corresponding to the current image block, and determines the block filtering weight corresponding to the current image block according to the motion information of the adjacent blocks; based on the pre-configured filter, determines the pixel filtering weight corresponding to each pixel in the current image block, and the block filtering weight and the pixel filtering weight are used for the time domain filtering processing of the video. On the one hand, by introducing the motion information of the adjacent blocks to adjust the block-level filtering weight of each image block in the video frame to be processed, the problem of excessive difference in block weights in time domain filtering can be avoided. On the other hand, based on the pre-configured filter, the pixel-level filtering weight corresponding to each pixel in the block can be optimized, which can avoid the problem of unreasonable weight amplitude difference being too large, which affects the filtering effect. On the other hand, through the above-mentioned weight optimization scheme, the optimized weight is applied to the subsequent time domain filtering process, which can optimize the time domain filtering effect of the video, thereby improving the overall coding efficiency.

[0088] Figure 6 FIG. 1 is a block diagram of a time domain filtering device according to an exemplary embodiment. Figure 6 The time domain filtering device 600 includes: an image block determination module 610, a block weight determination module 620 and a pixel weight determination module 630.

[0089] Specifically, the image block determination module 610 is used to obtain a video frame to be processed, which includes a current image block; the block weight determination module 620 is used to determine the adjacent blocks corresponding to the current image block, and determine the block filter weight corresponding to the current image block according to the motion information of the adjacent blocks; the pixel weight determination module 630 is used to determine the pixel filter weight corresponding to each pixel in the current image block based on a pre-configured filter, and the block filter weight and the pixel filter weight are used for time domain filtering processing of the video.

[0090] In an exemplary embodiment of the present disclosure, the block weight determination module 620 includes a block weight determination unit, which is used to: obtain a non-boundary image block in a video frame to be processed as a current image block; obtain multiple image blocks that are spatially adjacent to the current image block as adjacent blocks; determine block motion difference information between the adjacent blocks and the current image block based on motion information of the multiple adjacent blocks; and determine a block filtering weight based on the block motion difference information.

[0091] In an exemplary embodiment of the present disclosure, a block weight determination unit includes a difference information determination subunit, which is used to: determine a first motion vector corresponding to a current image block, the first motion vector including a first horizontal component and a first vertical component; determine a second motion vector corresponding to each of a plurality of adjacent blocks, the second motion vector including a second horizontal component and a second vertical component; and determine block motion difference information based on the first horizontal component, the first vertical component, the plurality of second horizontal components, and the plurality of second vertical components.

[0092] In an exemplary embodiment of the present disclosure, the block weight determination unit includes a block weight determination sub-unit, which is used to: determine the initial block filter weight corresponding to the current image block based on block motion difference information; obtain the frame type of the video frame to be processed, and determine the weight adjustment coefficient according to the frame type; determine the block filter weight according to the weight adjustment coefficient and the initial block filter weight.

[0093] In an exemplary embodiment of the present disclosure, the block weight determination subunit includes an initial block weight determination subunit, which is used to: obtain a pre-configured weight limiting function, the weight limiting function includes a first weight value and a second weight value; determine the block weight adjustment condition corresponding to the current image block based on block motion difference information; determine the initial block filtering weight according to the first weight value, the second weight value and the block weight adjustment condition.

[0094] In an exemplary embodiment of the present disclosure, the pixel weight determination module 630 includes a pixel weight determination unit, which is used to: obtain a pre-configured pixel filter and determine the filter weight and corresponding to the pixel filter; determine the initial pixel filter weight corresponding to each pixel based on the pixel filter and the filter weight; determine the pixel filter weight of each pixel in the block based on the initial pixel filter weight and the block filter weight.

[0095] In an exemplary embodiment of the present disclosure, the above-mentioned time domain filtering device 600 also includes a block weight configuration module, which is used to: obtain a boundary image block in a video frame to be processed; and configure the block filtering weight corresponding to the boundary image block to a preset weight value.

[0096] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0097] Reference below Figure 7 hereinafter describes an electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0098] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), and a display unit 740.

[0099] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0100] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 721 and / or a cache memory unit 722 , and may further include a read-only memory unit (ROM) 723 .

[0101] The storage unit 720 may include a program / utility 724 having a set (at least one) of program modules 725, such program modules 725 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0102] Bus 730 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0103] The electronic device 700 may also communicate with one or more external devices 770 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0104] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to perform the above-mentioned time domain filtering method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0105] In an exemplary embodiment, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the time domain filtering method described in any one of the above is implemented.

[0106] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0107] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A time domain filtering method, characterized in that: include: Acquire a video frame to be processed, where the video frame to be processed includes a current image block; Determine a neighboring block corresponding to the current image block, and determine a block filtering weight corresponding to the current image block according to motion information of the neighboring block; Based on a preconfigured filter, a pixel filter weight corresponding to each pixel in the current image block is determined, and the block filter weight and the pixel filter weight are used for time domain filtering processing of the video.

2. The method according to claim 1, characterized in that The step of determining the neighboring blocks corresponding to the current image block, and determining the block filtering weight corresponding to the current image block according to the motion information of the neighboring blocks, comprises: Acquire a non-boundary image block in the video frame to be processed as the current image block; Acquire a plurality of image blocks that are spatially adjacent to the current image block as the adjacent blocks; Determining block motion difference information between the adjacent blocks and the current image block according to motion information of the plurality of adjacent blocks; The block filter weight is determined based on the block motion difference information.

3. The method according to claim 2, characterized in that The determining, according to the motion information of the plurality of adjacent blocks, block motion difference information between the adjacent blocks and the current image block comprises: Determine a first motion vector corresponding to the current image block, where the first motion vector includes a first horizontal component and a first vertical component; Determine a second motion vector corresponding to each of the plurality of adjacent blocks, wherein the second motion vector includes a second horizontal component and a second vertical component; The block motion difference information is determined according to the first horizontal component, the first vertical component, a plurality of the second horizontal components, and a plurality of the second vertical components.

4. The method according to claim 2, characterized in that: The determining the block filtering weight based on the block motion difference information includes: Determining an initial block filter weight corresponding to the current image block based on the block motion difference information; Acquire the frame type of the video frame to be processed, and determine the weight adjustment coefficient according to the frame type; The block filter weight is determined according to the weight adjustment coefficient and the initial block filter weight.

5. The method according to claim 4, characterized in that The determining, based on the block motion difference information, an initial block filter weight corresponding to the current image block comprises: Obtaining a preconfigured weight clipping function, wherein the weight clipping function includes a first weight value and a second weight value; Based on the block motion difference information, determining a block weight adjustment condition corresponding to the current image block; The initial block filter weight is determined according to the first weight value, the second weight value and the block weight adjustment condition.

6. The method according to claim 1, characterized in that The determining, based on the preconfigured filter, a pixel filtering weight corresponding to each pixel in the current image block comprises: Obtain a preconfigured pixel filter and determine a filter weight and a corresponding filter weight of the pixel filter; Determining an initial pixel filter weight corresponding to each pixel according to the pixel filter and the filter weight sum; A pixel filter weight of each pixel in the block is determined according to the initial pixel filter weight and the block filter weight.

7. The method according to claim 1, characterized in that The method further comprises: Acquire a boundary image block in the video frame to be processed; The block filter weight corresponding to the boundary image block is configured as a preset weight value.

8. A time domain filtering device, characterized in that: include: An image block determination module, used to obtain a video frame to be processed, wherein the video frame to be processed includes a current image block; A block weight determination module, used to determine the adjacent blocks corresponding to the current image block, and determine the block filter weight corresponding to the current image block according to the motion information of the adjacent blocks; The pixel weight determination module is used to determine the pixel filter weight corresponding to each pixel in the current image block based on a pre-configured filter, and the block filter weight and the pixel filter weight are used for time domain filtering processing of the video.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the time domain filtering method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the time domain filtering method according to any one of claims 1 to 7 is implemented.

Images