Time domain filtering method and system

Abstract

The embodiment of the invention provides a time domain filtering method and system. The method is used for carrying out time domain filtering on each image frame in a video, and comprises the following steps: obtaining a current frame and a reference frame from the video, and dividing the current frame into a plurality of image blocks; for each image block in the current frame, determining motion information of the image block and motion information of N adjacent blocks of the image block by taking the reference frame as a reference, and determining a prediction block corresponding to the image block based on the motion information of the image block, the motion information of the N adjacent blocks and the reference frame, filtering the image block based on the prediction block to obtain a filtering result of the image block; determining the filtering result of the current frame based on the filtering result of each image block; wherein the motion information of the image block represents the position offset of the content of the image block between the current frame and the reference frame, the motion information of each adjacent block represents the position offset of the content of the adjacent block between the current frame and the reference frame, and N is an integer greater than or equal to 1.

Description

Technical Field

[0001] The present specification relates to the field of multimedia technology, and in particular to a time domain filtering method and system. Background Art

[0002] With the increase of Internet bandwidth and the explosive growth of video content consumption, high-definition video and ultra-high-definition video (such as 4K, 8K) have become a trend. These high-quality video contents bring a more realistic visual experience, but also bring huge challenges to video service platforms. In order to maintain high video quality while reducing bandwidth usage and storage costs, video encoding technology is particularly important.

[0003] However, the noise in the video is an important factor affecting the coding efficiency. Therefore, before encoding the video, it is usually necessary to filter the video to remove the noise in the video. How to filter the video without reducing the visual effect or reducing it slightly, so as to maximize the efficiency of subsequent video coding, is still a problem that needs to be solved.

[0004] The content of the background technology section is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the invention

[0005] The present specification provides a time domain filtering method and system, which can remove noise in a video without reducing or minimizing the visual effect, so as to improve the efficiency of subsequent video encoding.

[0006] In a first aspect, the present specification provides a time domain filtering method for performing time domain filtering on each image frame in a video, comprising: obtaining a current frame and a reference frame from the video, wherein the current frame is divided into multiple image blocks; for each image block in the current frame, using the reference frame as a reference, determining motion information of the image block and motion information of N adjacent blocks of the image block, wherein the motion information of the image block represents a position offset of a content of the image block between the current frame and the reference frame, and the motion information of each adjacent block represents a position offset of a content of the adjacent block between the current frame and the reference frame, wherein N is an integer greater than or equal to 1, determining a prediction block corresponding to the image block based on the motion information of the image block, the motion information of the N adjacent blocks and the reference frame, filtering the image block through the prediction block to obtain the filtered image block; and determining the filtered current frame based on the multiple filtered image blocks.

[0007] In some embodiments, determining the prediction block corresponding to the image block based on the motion information of the image block, the motion information of the N neighboring blocks and the reference frame includes: determining a first reference image block matching the content of the image block in the reference frame based on the motion information of the image block and the position of the image block in the current frame; determining M target neighboring blocks from the N neighboring blocks, and for each target neighboring block, determining a second reference image block matching the content of the target neighboring block in the reference frame based on the motion information of the target neighboring block and the position of the image block in the current frame, to obtain a total of M second reference image blocks, where M is an integer less than or equal to N; and determining the prediction block corresponding to the image block based on the first reference image block and the M second reference image blocks.

[0008] In some embodiments, determining the prediction block corresponding to the image block based on the first reference image block and the M second reference image blocks includes: fusing the first reference image block and the M second reference image blocks to obtain a fused image block, and using the fused image block as the prediction block.

[0009] In some embodiments, determining the prediction block corresponding to the image block based on the first reference image block and the M second reference image blocks includes: fusing the first reference image block and the M second reference image blocks to obtain a fused image block; and selecting the one with the smallest difference from the image block from the fused image block and the first reference image block as the prediction block.

[0010] In some embodiments, the M target adjacent blocks include a left adjacent block and an upper adjacent block, the M second reference image blocks include: a left reference image block determined for the left adjacent block and an upper reference image block determined for the upper adjacent block, and the fusing of the first reference image block and the M second reference image blocks to obtain a fused image block includes: fusing the first reference image block with the left reference image block to obtain an intermediate fused image block; and fusing the intermediate fused image block with the upper reference image block to obtain the fused image block.

[0011] In some embodiments, the fusing of the first reference image block and the left reference image block to obtain an intermediate fused image block includes: weightedly fusing the left half area of ​​the left reference image block to the left half area of ​​the first reference image block by pixel column to obtain the intermediate fused image block, wherein the total number of pixel columns of the left reference image block or the first reference image block is W, the weight corresponding to the i-th column of pixels in the left half area of ​​the left reference image block is positively correlated with i, the weight corresponding to the i-th column of pixels in the left half area of ​​the first reference image block is negatively correlated with i, and i is less than or equal to W / 2.

[0012] In some embodiments, the intermediate fused image block is obtained based on the following formula:

[0013]

[0014] Among them, P′(i,j) represents the pixel in the jth row and i-th column in the intermediate fused image block, P(i,j) represents the pixel in the jth row and i-th column in the first reference image block, w(i) represents the weight corresponding to the pixel in the i-th column in the left reference image block, LP(i,j) represents the pixel in the jth row and i-th column in the left reference image block, and a is a preset value.

[0015] In some embodiments, fusing the intermediate fused image block with the upper reference image block to obtain the fused image block includes: weightedly fusing the upper half area of ​​the upper reference image block to the upper half area of ​​the intermediate fused image block by pixel rows to obtain the fused image block, wherein the total number of pixel rows of the upper reference image block or the intermediate fused image block is H, the weight corresponding to the j-th row of pixels in the upper half area of ​​the upper reference image block is positively correlated with j, the weight corresponding to the j-th row of pixels in the upper half area of ​​the intermediate fused image block is negatively correlated with j, and j is less than or equal to H / 2.

[0016] In some embodiments, the fused image block is obtained based on the following formula:

[0017]

[0018] Among them, P″(i,j) represents the pixel in the jth row and i-th column of the fused image block, P′(i,j) represents the pixel in the jth row and i-th column of the intermediate fused image block, w(j) represents the weight corresponding to the pixel in the jth row of the upper reference image block, TP(i,j) represents the pixel in the jth row and i-th column of the upper reference image block, and b is a preset value.

[0019] In some embodiments, determining M target adjacent blocks from the N adjacent blocks includes: for each adjacent block among the N adjacent blocks, if the difference between the motion information of the adjacent block and the motion information of the image block is within a preset range, determining the adjacent block as the target adjacent block.

[0020] In some embodiments, the difference between the motion information of the adjacent block and the motion information of the image block is within a preset range, including at least one of the following: the difference between the horizontal component information in the motion information of the adjacent block and the horizontal component information in the motion information of the image block is within a first preset range; or the difference between the vertical component information in the motion information of the adjacent block and the vertical component information in the motion information of the image block is within a second preset range.

[0021] In some embodiments, the method further includes: performing hierarchical motion estimation on the current frame and the reference frame to obtain a motion estimation result, wherein the motion estimation result includes position information of each image block in the current frame and motion information of each image block; determining the motion information of the image block and the motion information of N adjacent blocks of the image block includes: querying and obtaining the motion information of the image block from the motion estimation result based on the position information of the image block in the current frame; and for each adjacent block, determining the position information of the adjacent block in the current frame based on the position information of the image block in the current frame and the positional relationship between the image block and the adjacent block, and querying and obtaining the motion information of the adjacent block from the motion estimation result based on the position information of the adjacent block in the current frame.

[0022] In some embodiments, the N neighboring blocks include at least one of an upper neighboring block, a lower neighboring block, a left neighboring block, and a right neighboring block.

[0023] In a second aspect, the present specification also provides a time domain filtering system, comprising: at least one storage medium storing at least one instruction set for performing time domain filtering; and at least one processor communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set during operation and implements the method described in the first aspect according to the instructions of the at least one instruction set.

[0024] Other functions of the time domain filtering method and system provided in this specification will be partially listed in the following description. The creative aspects of the time domain filtering method and system provided in this specification can be fully explained by practicing or using the methods, systems and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of an application scenario of time domain filtering provided according to an embodiment of this specification is shown;

[0027] Figure 2 A hardware structure diagram of a time domain filtering system provided according to an embodiment of this specification is shown;

[0028] Figure 3 A flow chart of a time domain filtering method provided according to an embodiment of this specification is shown;

[0029] Figure 4 A schematic diagram showing the positions of adjacent blocks provided according to an embodiment of this specification is shown;

[0030] Figure 5 A schematic diagram showing the position of an image block in a current frame according to an embodiment of this specification;

[0031] Figure 6 A flowchart for determining a prediction block according to an embodiment of this specification is shown;

[0032] Figure 7 A schematic diagram showing the position of a first reference image block provided according to an embodiment of this specification;

[0033] Figure 8 A schematic diagram showing the position of a second reference image block provided according to an embodiment of this specification;

[0034] Fig. 9 A schematic diagram showing a method of determining a prediction block by way of method 1 according to an embodiment of this specification is shown;

[0035] Fig.10 A schematic diagram of determining a prediction block by way of method 2 according to an embodiment of this specification is shown;

[0036] Fig.11 A schematic diagram showing row pixels and column pixels provided according to an embodiment of the present specification; and

[0037] Fig.12 A schematic diagram showing the difference in weights of different pixel columns provided according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0038] The following description provides specific application scenarios and requirements of this specification, with the purpose of enabling those skilled in the art to make and use the contents of this specification. Various local modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but to the widest scope consistent with the claims.

[0039] The terms used herein are only used for the purpose of describing specific example embodiments and are not restrictive. For example, unless the context clearly indicates otherwise, as used herein, the singular forms "a", "an" and "the" may also include plural forms. When used in this specification, the terms "include", "comprise" and / or "contain" mean that the associated integers, steps, operations, elements and / or components exist, but do not exclude the existence of one or more other features, integers, steps, operations, elements, components and / or groups or that other features, integers, steps, operations, elements, components and / or groups may be added in the system / method.

[0040] In view of the following description, these and other features of the present specification, as well as the operation and function of the related elements of the structure, and the economy of the combination and manufacture of the parts can be significantly improved. Reference is made to the accompanying drawings, all of which form a part of this specification. However, it should be clearly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0041] The flowcharts used in this specification illustrate the operations implemented by the system according to some embodiments in this specification. It should be clearly understood that the operations of the flowcharts may not be implemented in sequence. On the contrary, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0042] For ease of understanding, this specification provides the following explanations for the terms that will appear below.

[0043] Motion Estimation (ME): Divide the image frames in the video into multiple image blocks, search for the position of each image block in each image frame (current frame) in the reference frame to describe the motion of the content of the image block in the image frame. For example, according to the matching criteria, find the image block in the reference frame that is most similar to the content of the current block of the current frame, which is called the matching block. The relative displacement between the matching block and the current block is the motion information (such as a motion vector), which can represent the motion of the content of the current block between the current frame and the reference frame.

[0044] Hierarchical Motion Estimation (HME): is a technique to optimize video coding efficiency by performing motion estimation at different levels. HME usually involves motion estimation at multiple resolution levels or "levels", starting from lower resolution levels (or coarser levels) and then gradually transitioning to higher resolution levels (or finer levels). At each level, motion search and matching are performed to find matching blocks that meet the conditions and then determine motion information, thereby describing the movement of image blocks between image frames.

[0045] Motion Compensation (MC): It is a video processing technology that adjusts and predicts image frames through motion information obtained by motion estimation, which can reduce the difference between image frames and redundant information in the video. The purpose of motion compensation is to improve compression efficiency and video quality by accurately predicting the content of image frames during video encoding and compression.

[0046] Motion Compensated Temporal Filter (MCTF) is a video processing technology that uses the motion information between image frames in a video to filter the image frames to remove the noise between the image frames, thereby improving the efficiency of video encoding. Temporal filtering is a preprocessing technology before video encoding. After obtaining the filtered video, the encoder can encode and compress it.

[0047] In some embodiments, during the process of time domain filtering of a video, for each image block of a current frame in the video, a motion vector of the image block is determined with a reference frame as a reference, and a prediction block corresponding to the image block is determined based on the motion information of the image block and the reference frame, the image block is filtered through the prediction block, and then the filtered current frame is determined based on each filtered image block.

[0048] In the above-mentioned time domain filtering process, the image frame is divided into image blocks, and filtering is performed with the image blocks as the granularity. The boundaries of these image blocks become obvious and not smooth after filtering, thereby forming an unnatural block structure visually, that is, causing a block effect. Although the noise in the video is removed, the visual effect is poor.

[0049] In view of this, the present specification provides a time domain filtering method and system, so as to reduce the block effect while removing noise in the video, without reducing or minimizing the visual effect.

[0050] Combine the following Figure 1 The application scenarios of time domain filtering provided in this specification are introduced.

[0051] Figure 1 FIG. 2 shows a schematic diagram of an application scenario of time domain filtering provided according to an embodiment of this specification. Figure 1 As shown, scene 001 may include a time domain filtering system 100 and a coding system 200. The time domain filtering system 100 is used to perform time domain filtering on the original video, and the coding system 200 is used to encode the filtered video.

[0052] The time domain filtering system 100 and the encoding system 200 may correspond to the same system or to different systems, and this specification does not limit this. For example, the time domain filtering system 100 and the encoding system 200 are both located in an encoder, and the encoder is, for example, an encoder that executes the AV1 (AO Media Video 1) video encoding format, or an encoder that executes other video encoding formats such as AVC (Advanced Video Coding), HEVC (High Efficiency Video Coding), VVC or VP9, ​​and this specification does not limit this.

[0053] Exemplarily, the temporal filtering system 100 includes a motion estimation module, a motion compensation module and a filtering module. The motion estimation module is used to perform motion estimation on each image frame in the original video by using the HME technology, so as to obtain motion information (such as motion vector) of each image block in each image frame. The motion compensation module is used to predict each image block by using the motion information of the motion estimation to obtain prediction information (such as prediction block, etc.). The filtering module is used to filter each image block by using the prediction information.

[0054] Understandably, Figure 1 The module division of the time domain filtering system 100 is only an example, and the present specification is not limited thereto.

[0055] In some embodiments, the time domain filtering system 100 may store data and instructions for implementing the time domain filtering method, and may execute or be used to execute the data and instructions. In some embodiments, the time domain filtering system 100 may include a hardware device with a data information processing function and a necessary program required to drive the hardware device to work.

[0056] Exemplarily, in the process of performing time-domain filtering on each image frame in the original video, the time-domain filtering system 100 obtains a current frame and a reference frame from the original video, and the current frame is divided into multiple image blocks. For each image block in the current frame, the motion estimation module determines the motion information of the image block and the motion information of N adjacent blocks of the image block with the reference frame as a reference, wherein the motion information of the image block represents the position offset of the content of the image block between the current frame and the reference frame, and the motion information of each adjacent block represents the position offset of the content of the adjacent block between the current frame and the reference frame, and N is an integer greater than or equal to 1; the motion compensation module determines the prediction block corresponding to the image block based on the motion information of the image block, the motion information of the N adjacent blocks and the reference frame; the filtering module filters the image block through the prediction block to obtain the filtered image block, and then determines the filtered current frame based on the filtered multiple image blocks.

[0057] It should be noted that the time domain filtering system 100 may correspond to one device or to a device cluster, and this specification does not limit this. When the time domain filtering system 100 corresponds to one device, the time domain filtering method may be executed on the device. When the time domain filtering system 100 corresponds to a device cluster, the time domain filtering method may be executed in a distributed manner on multiple devices corresponding to the device cluster, or the time domain filtering method may have other execution modes, and this specification does not limit this.

[0058] Figure 2 FIG. 1 shows a hardware structure diagram of a time-domain filtering system 100 provided according to an embodiment of the present specification.

[0059] like Figure 2 As shown, the time domain filtering system 100 may include at least one storage medium 130 and at least one processor 120. In some embodiments, the time domain filtering system 100 may further include a communication port 150 and an internal communication bus 110. In addition, the time domain filtering system 100 may further include an I / O component 160.

[0060] The internal communication bus 110 can connect different system components. For example, the internal communication bus 110 can connect the storage medium 130, the processor 120, the communication port 150 and the I / O component 160.

[0061] The I / O component 160 supports input / output between the time-domain filtering system 100 and other components.

[0062] The communication port 150 is used for data communication between the time domain filtering system 100 and the outside world. For example, the communication port 150 can be used for data communication between the time domain filtering system 100 and a network. The communication port 150 can be a wired communication port or a wireless communication port.

[0063] In some embodiments, the network may be any type of wired or wireless network, or a combination thereof. For example, the network may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network TM, a short-range wireless network (ZigBee TM), a near field communication (NFC) network, or a similar network.

[0064] In some embodiments, the network may include one or more network access points. For example, the network may include a wired or wireless network access point, such as a base station or an Internet exchange point. Through the access point, one or more components of each device corresponding to the time domain filtering system 100 can be connected to the network to exchange data or information.

[0065] The storage medium 130 may include a data storage device. The data storage device may be a non-temporary storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 132, a read-only storage medium (ROM) 134, or a random access storage medium (RAM) 136. The storage medium 130 also includes at least one instruction set stored in the data storage device. The instruction set may include a computer program code, and the computer program code may include a program, a routine, an object, a component, a data structure, a process, a module, etc. for executing the time domain filtering method provided in this specification.

[0066] The processor 120 may be in communication with the storage medium 130. The processor 120 is used to execute the at least one instruction set. When the time domain filtering system 100 is running, the processor 120 reads the at least one instruction set and executes the time domain filtering method provided in this specification according to the instruction of the at least one instruction set.

[0067] The processor 120 may be in the form of one or more processors. In some embodiments, the processor 120 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, etc., or any combination thereof.

[0068] For illustration purposes only, only one processor 120 is shown in the time domain filtering system 100 in the accompanying drawings. However, it should be noted that the time domain filtering system 100 in this specification may also include multiple processors. Therefore, the operations and / or method steps disclosed in this specification may be performed by one processor as described in this specification, or may be performed jointly by multiple processors. For example, if the processor 120 of the time domain filtering system 100 in this specification performs step A and step B, it should be understood that step A and step B may also be performed jointly or separately by two different processors 120 (for example, the first processor performs step A, the second processor performs step B, or the first and second processors perform steps A and B together).

[0069] Figure 3 A flow chart of a time domain filtering method P300 provided according to an embodiment of the present specification is shown. The time domain filtering system 100 can execute the time domain filtering method P300 to perform time domain filtering on each image frame in a video.

[0070] like Figure 3 As shown, the time domain filtering method P300 includes the following steps.

[0071] S310: Obtain a current frame and a reference frame from a video, and divide the current frame into a plurality of image blocks.

[0072] The video may be a video to be transmitted, or a video to be encoded. The current frame is one of the image frames to be filtered in the video. The reference frame may be an image frame adjacent to the current frame (either directly or indirectly) during video playback or video encoding, such as an I frame, a P frame, or a B frame, and this specification does not limit this. The reference frame may be one or more, and this specification does not limit this. When there are multiple reference frames, the process based on the current frame and each reference frame is the same. The following text takes one of the reference frames as an example to introduce the process of filtering multiple image blocks in the current frame by the time domain filtering system 100. The execution process using other reference frames as references is similar and will not be repeated.

[0073] The multiple image blocks may be of the same size and may not overlap each other, and this specification does not impose any limitation on this.

[0074] The time domain filtering system 100 filters each image frame in the video to obtain a filtered video. The filtering method of each image frame in the video by the time domain filtering system 100 is similar. The filtering of the current frame is used as an example for explanation below, and the filtering process of other image frames is not repeated.

[0075] For each image block in the current frame, the filtering process is performed with the reference frame as a reference. The filtering process of image block #1 by the time domain filtering system 100 is introduced below by taking image block #1 as an example. The filtering processes of other image blocks are similar and will not be described in detail. The filtering process of image block #1 includes S320, S330 and S340.

[0076] S320: Determine the motion information of image block #1 and the motion information of N adjacent blocks of image block #1, wherein the motion information of image block #1 represents the position offset of the content of image block #1 between the current frame and the reference frame, the motion information of each adjacent block represents the position offset of the content of the adjacent block between the current frame and the reference frame, and N is an integer greater than or equal to 1.

[0077] Among them, the adjacent block refers to the image block adjacent to the image block #1 in the current frame. In some embodiments, the N adjacent blocks include at least one of the upper adjacent block, the lower adjacent block, the left adjacent block, and the right adjacent block. Generally speaking, there is motion correlation between adjacent image blocks. Therefore, considering the motion information of the adjacent blocks bordering the image content of the image block #1 during the filtering process can obtain more accurate motion compensation results, smoothly transition the boundaries of the image blocks, thereby reducing the generation of block effects and improving the quality of the filtered video.

[0078] For example, Figure 4The current frame after division is shown, where the size of each small block is 32x32. If the size of image block #1 is 64x64, and image block #1 corresponds to the blocks composed of C0, C1, C2 and C3, then the upper neighboring block of image block #1 is the block composed of T0, T1, T2 and T3, the left neighboring block is the block composed of L0, L1, L2 and L3, and the lower and right neighboring blocks of image block #1 are similar. If the size of image block #1 is 32x32, and image block #1 corresponds to C0, then the upper neighboring block of image block #1 is T2, the left neighboring block is L1, the lower neighboring block is C2, and the right neighboring block is C1.

[0079] like Figure 5 As shown, when image block #1 is located at position #1 of the current frame, the N adjacent blocks of image block #1 include the right adjacent block and the lower adjacent block; when image block #1 is located at position #2 of the current frame, the N adjacent blocks of image block #1 include the left adjacent block, the upper adjacent block, the right adjacent block and the lower adjacent block; when image block #1 is located at position #3 of the current frame, the N adjacent blocks of image block #1 include the left adjacent block and the upper adjacent block. That is, when image block #1 is located at different positions of the current frame, the N adjacent blocks of image block #1 are different.

[0080] In the process of filtering each image frame in the video, the present specification can adaptively determine the filtering granularity, that is, the block size of the filtered image block, and can also adaptively select adjacent blocks for motion compensation, thereby improving the accuracy of motion compensation.

[0081] In some embodiments, before S320, the temporal filtering method P300 further includes: performing hierarchical motion estimation on the current frame and the reference frame to obtain a motion estimation result. The motion estimation result includes position information of each image block in the current frame and motion information of each image block.

[0082] It can be understood that hierarchical motion estimation also needs to divide the image frame into blocks. It should be noted that the hierarchical motion estimation blocks and the filtering blocks are decoupled, that is, the sizes of the hierarchical motion estimation blocks and the filtering blocks can be different. The size of the hierarchical motion estimation blocks can be 64x64, 32x32, 16x16 or 8x8. In the following text, the blocks divided into the current frame during hierarchical motion estimation are referred to as estimation blocks, and the names are distinguished from the image blocks divided into the current frame during filtering.

[0083] Before performing hierarchical motion estimation, the temporal filtering system 100 may initialize a data table describing the motion estimation result for each estimation block.

[0084] Taking the estimation block #1 in the current frame as an example, the data structure of the data table is as follows:

[0085] typedef struct NeighborMeContext{

[0086] uint32_t frame_idx;

[0087] uint32_t blk_x;

[0088] uint32_t blk_y;

[0089] signed short tf_8x8_mv_x

[64] ;

[0090] signed short tf_8x8_mv_y

[64] ;

[0091] signed short tf_16x16_mv_x

[16] ;

[0092] signed short tf_16x16_mv_y

[16] ;

[0093] signed short tf_32x32_mv_x[4];

[0094] signed short tf_32x32_mv_y[4];

[0095] signed short tf_64x64_mv_x;

[0096] signed short tf_64x64_mv_y;

[0097] }NeighborMeContext;

[0098] Wherein, 'frame_idx' indicates the serial number of the current frame; 'blk_x' indicates the horizontal position information of the estimated block #1 in the current frame, 'blk_y' indicates the vertical position information of the image block #1 in the current frame; 'signed short tf_64x64_mv_x' indicates the horizontal motion information of the estimated block #1, 'signed short tf_64x64_mv_y' indicates the vertical motion information of the estimated block #1; 'signed short tf_32x32_mv_y[4]' indicates the horizontal motion information of each sub-block after the estimated block #1 is divided into 4 sub-blocks of size 32x32, 'signed short tf_32x32_mv_x[4]' indicates the vertical motion information of each sub-block after the estimated block #1 is divided into 4 sub-blocks of size 32x32; 'signed short tf_16x16_mv_x

[16] ' indicates the horizontal motion information of each sub-block after the estimation block #1 is divided into 16 sub-blocks of size 16x16, 'signed short tf_16x16_mv_y

[16] ' indicates the vertical motion information of each sub-block after the estimation block #1 is divided into 16 sub-blocks of size 16x16; 'signed short tf_8x8_mv_x

[64] ' and 'signed short tf_8x8_mv_y

[64] ' are understood in the same way. During initialization, all motion information can be set to 0. The horizontal motion information can be a horizontal position offset, and the vertical motion information can be a vertical position offset.

[0099] Exemplarily, when the temporal filtering system 100 performs hierarchical motion estimation, it constructs image pyramids of different resolution levels for the current frame and the reference frame, respectively, wherein the low-resolution layer is obtained by downsampling the high-resolution layer image. The temporal filtering system 100 performs preliminary motion estimation from the lowest resolution layer to obtain preliminary motion information, uses the preliminary motion information as the initial estimation of the higher resolution layer, refines the motion information at the higher resolution layer, that is, further searches and optimizes within a smaller range under the guidance of the initial motion information to obtain more accurate motion information; repeats the above steps until the highest resolution layer (original resolution layer) is reached, and completes the final motion estimation at the highest resolution layer. The temporal filtering system 100 determines the motion information of each estimation block based on the final motion estimation, updates the motion information of each estimation block to the data table corresponding to each estimation block, and stores it as the motion estimation result of the hierarchical motion estimation.

[0100] After filtering is started for image block #1, the time domain filtering system 100 can query the motion information of image block #1 from the motion estimation result based on the position information of image block #1 in the current frame. For each adjacent block, the time domain filtering system 100 can determine the position information of each adjacent block in the current frame based on the position information of image block #1 in the current frame and the position relationship between image block #1 and each adjacent block, and query the motion information of each adjacent block from the motion estimation result based on the position information of each adjacent block in the current frame.

[0101] Exemplarily, after filtering is turned on for image block #1, the time domain filtering system 100 may select a data table of an estimated block corresponding to image block #1 from the stored motion estimation results based on the position information of image block #1 in the current frame, and determine the motion information of image block #1 from the data table. For adjacent blocks, the time domain filtering system 100 determines the position information of the adjacent blocks in the current frame based on the positional relationship between image block #1 and the adjacent blocks, selects a data table of an estimated block corresponding to the adjacent blocks from the stored motion estimation results based on the position information, and copies the motion information of each adjacent block from the data table.

[0102] The time domain filtering system 100 obtains relatively accurate motion information of each estimated block in the current frame through hierarchical motion estimation, which can improve the search efficiency of motion estimation. In addition, the motion information of each estimated block is stored before filtering. The time domain filtering system 100 can directly obtain the motion information corresponding to each image block from the stored information during filtering, thereby improving the efficiency of obtaining motion information.

[0103] S330: Determine a prediction block corresponding to the image block #1 based on the motion information of the image block #1, the motion information of the N adjacent blocks and the reference frame.

[0104] The prediction block can be understood as the image block that is searched from the reference frame and is most matched with the image block #1, or the image block that is selected from the reference frame and has the smallest image content difference with the image block #1.

[0105] In some embodiments, Figure 5 As shown, S330 includes S3302, S3304 and S3306. In addition to determining the reference image block based on the motion information of image block #1, the time domain filtering system 100 can also determine other reference image blocks based on the motion information of each adjacent block, so that there are more reference options when determining the prediction block, which is conducive to improving the accuracy of the obtained prediction block.

[0106] S3302: Based on the motion information of the image block #1 and the position of the image block #1 in the current frame, determine a first reference image block in the reference frame that matches the content of the image block #1.

[0107] Exemplarily, the time domain filtering system 100 determines a first position in a reference frame based on horizontal motion information and vertical motion information of image block #1 and the position of image block #1 in a current frame, and uses the image content corresponding to the first position as a first reference image block, and the first reference image block has the same size as image block #1.

[0108] For example, the position of image block #1 in the current frame is Figure 7 As shown, it is assumed that the horizontal motion information of the image block #1 is one image block width to the right, and the vertical motion information is one image block height to the bottom, that is, the position of the first reference image block in the reference frame is shown in FIG. Figure 7 .

[0109] S3304: Determine M target adjacent blocks from the N adjacent blocks, and for each target adjacent block, determine a second reference image block in the reference frame that matches the content of the target adjacent block based on the motion information of the target adjacent block and the position of image block #1 in the current frame, and obtain a total of M second reference image blocks, where M is an integer less than or equal to N.

[0110] In some embodiments, for each of the N neighboring blocks, if the difference between the motion information of the neighboring block and the motion information of image block #1 is within a preset range, the time domain filtering system 100 determines the neighboring block as the target neighboring block.

[0111] When the motion information of a certain neighboring block differs greatly from the motion information of image block #1, the time domain filtering system 100 does not consider the motion information of the neighboring block when determining the prediction block corresponding to image block #1, thereby ensuring the accuracy of subsequent motion compensation. For example, when the motion information of the upper neighboring block differs greatly from the motion information of image block #1, the time domain filtering system 100 does not consider the motion information of the upper neighboring block when determining the prediction block corresponding to image block #1.

[0112] Exemplarily, the difference between the motion information of the adjacent blocks and the motion information of the image block #1 is within a preset range, which is classified into at least one of the following situations:

[0113] The difference between the horizontal component information in the motion information of the adjacent block and the horizontal component information in the motion information of the image block #1 is within a first preset range; or

[0114] The difference between the vertical component information in the motion information of the adjacent block and the vertical component information in the motion information of the image block #1 is within a second preset range.

[0115] The horizontal component information can be understood as horizontal motion information, and the vertical component information can be understood as vertical motion information. That is, when the horizontal motion information of the adjacent block differs greatly from the horizontal motion information of the image block #1, or the vertical motion information of the adjacent block differs greatly from the vertical motion information of the image block #1, the time domain filtering system 100 does not consider the motion information of the adjacent block when determining the prediction block corresponding to the image block #1, so as to avoid affecting the accuracy of the motion compensation of the image block #1 when the horizontal motion information differs greatly or the vertical motion information differs greatly. The first preset range and the second preset range can be the same or different, and this specification does not limit this.

[0116] If only the motion information of the upper neighboring block and the left neighboring block among the N adjacent blocks differs from the motion information of the image block #1 within a preset range, then the target adjacent blocks only include the upper neighboring block and the left neighboring block.

[0117] It can be understood that when the time domain filtering system 100 determines the target adjacent block from the N adjacent blocks, it can be executed in a preconfigured order, for example, first determining whether the left adjacent block is the target adjacent block, then determining whether the upper adjacent block is the target adjacent block, and then determining whether other adjacent blocks are the target adjacent blocks. This specification does not limit the order.

[0118] After the time domain filtering system 100 determines the target adjacent block, taking the upper adjacent block as an example, a second reference image block matching the content of the upper adjacent block can be determined in the reference frame based on the motion information of the upper adjacent block and the position of image block #1 in the current frame.

[0119] For example, the position of image block #1 in the current frame is as follows: Figure 7 As shown, assuming that the horizontal motion information of the upper adjacent block is the width of an image block to the right, and the vertical motion information is the height of an image block below, the position of the second reference image block in the reference frame is the same as the position of the first reference image block in the reference frame.

[0120] Assuming that the horizontal motion information of the upper neighboring block is one image block width to the right, but the vertical motion information is 0.5 image block height downward, the position of the second reference image block in the reference frame is as follows: Figure 8 When the target adjacent block includes the left adjacent block, the time domain filtering system 100 determines in the reference frame that the second reference image block matching the content of the left adjacent block is similar and will not be described in detail.

[0121] S3306: Determine a prediction block corresponding to image block #1 based on the first reference image block and the M second reference image blocks.

[0122] There are many ways for the time domain filtering system 100 to determine the prediction block corresponding to the image block #1. Two of them are exemplified below, and this specification does not limit them.

[0123] Method 1

[0124] In some embodiments, Fig. 9 As shown, the time domain filtering system 100 fuses the first reference image block and the M second reference image blocks to obtain a fused image block, and uses the fused image block as the prediction block. This can reduce the block effect during the filtering process and improve the smoothness of the content in the filtered image frame.

[0125] The following describes the method of obtaining the fused image block by the time domain filtering system 100 by taking the M target adjacent blocks including the left adjacent block and the upper adjacent block as an example. When the M target adjacent blocks include other adjacent blocks, the method of obtaining the fused image block is similar and will not be described in detail. When the target adjacent blocks include multiple blocks, the smoothness of the image content near more edges of the filtered image block #1 can be improved.

[0126] When the M target neighboring blocks include a left neighboring block and an upper neighboring block, the M second reference image blocks include: a left reference image block determined based on motion information of the left neighboring block and an upper reference image block determined based on motion information of the upper neighboring block.

[0127] The time domain filtering system 100 fuses the first reference image block with the left reference image block to obtain an intermediate fused image block, and fuses the intermediate fused image block with the upper reference image block to obtain the fused image block.

[0128] Exemplarily, in the process of fusing the first reference image block and the left reference image block, the time domain filtering system 100 performs weighted fusion on the first reference image block and the left reference image block according to pixel columns to obtain the intermediate fused image block.

[0129] The weight corresponding to the pixels in the i-th column in the left reference image block is inversely correlated with the weight corresponding to the pixels in the i-th column in the first reference image block.

[0130] For example, the intermediate fused image block is obtained based on the formula of the following example:

[0131]

[0132] Among them, P′(i,j) represents the pixel in the jth row and ith column in the intermediate fused image block, P(i,j) represents the pixel in the jth row and ith column in the first reference image block, w(i) represents the weight corresponding to the pixel in the i-th column in the left reference image block, LP(i,j) represents the pixel in the jth row and i-th column in the left reference image block, a is a preset value, W represents the width of the image block or the total number of pixel columns of the left reference image block or the first reference image block, 0≤i≤W, 0≤j≤H, H represents the width of the image block or the total number of pixel rows of the left reference image block or the first reference image block.

[0133] It can be understood that the first reference image block, the left reference image block and the intermediate fused image block have the same block size. a is, for example, 64 or other values, which is not limited in this specification.

[0134] Taking the first reference image block as an example, the pixels in the i-th column and the j-th row are as follows: Fig.11 As shown, the pixels in the ith column and the jth row of the left reference image block are similar and will not be described in detail.

[0135] Specifically, during the fusion process, the time domain filtering system 100 may only fuse the first reference image block and the left half area of ​​the left reference image block, that is, during the fusion process through the above formula, The weight corresponding to the i-th column of pixels in the left half of the left reference image block is positively correlated with i, and the weight corresponding to the i-th column of pixels in the left half of the first reference image block is negatively correlated with i. That is, the left half of the image content of image block #1 is smoothed by the left half of the image content of the left reference image block. Of course, there are other fusion methods, which are not listed here. In the above fusion process, the weight difference and weight change trend of pixels in different columns in the left half of the left reference image block and the first reference image block are shown in Fig.12 .

[0136] During the fusion process, the time domain filtering system 100 sets different weights for different columns of pixels according to the influence of different columns of pixels, so that a more accurate motion compensation result can be obtained by fusion.

[0137] Exemplarily, after obtaining the intermediate fused image block, the time domain filtering system 100 performs weighted fusion of the intermediate fused image block and the upper reference image block by pixel rows during the process of fusing the intermediate fused image block with the upper reference image block to obtain the fused image block.

[0138] The weight corresponding to the j-th row of pixels in the upper reference image block is inversely correlated with the weight corresponding to the j-th row of pixels in the intermediate fused image block.

[0139] For example, the fused image block is obtained based on the formula shown below:

[0140]

[0141] Wherein, P″(i,j) represents the pixel in the jth row and ith column of the fused image block, P′(i,j) represents the pixel in the jth row and ith column of the intermediate fused image block, w(j) represents the weight corresponding to the pixel in the jth row of the upper reference image block, TP(i,j) represents the pixel in the jth row and ith column of the upper reference image block, b is a preset value, W represents the width of the image block or the total number of pixel columns of the upper reference image block or the intermediate fused image block, 0≤i≤W, 0≤j≤H, H represents the width of the image block or the total number of pixel rows of the upper reference image block or the intermediate fused image block.

[0142] It can be understood that the block size of the intermediate fused image block, the upper reference image block and the fused image block is the same. b is, for example, 64 or other values, which are not limited in this specification. The j-th row pixel and the i-th column pixel in the intermediate fused image block and the upper reference image block are Fig.11 The instructions are similar and will not be repeated here.

[0143] Specifically, during the fusion process, the time domain filtering system 100 may fuse only the upper half of the upper reference image block and the intermediate fused image block, that is, during the fusion process through the above formula, The weight corresponding to the j-th row of pixels in the upper half of the upper reference image block is positively correlated with j, and the weight corresponding to the j-th row of pixels in the upper half of the intermediate fused image block is negatively correlated with j. That is, the upper half of the image content of image block #1 is smoothed by the upper half of the image content of the upper reference image block. Of course, there can be other fusion methods, which are not enumerated. In the above fusion process, the weight difference and weight change trend of different rows of pixels in the upper half of the upper reference image block and the intermediate fused image block are the same as Fig.12 Similar, no more hints.

[0144] During the fusion process, the time domain filtering system 100 sets different weights for different columns of pixels according to the influence of different columns of pixels, so that a more accurate motion compensation result can be obtained by fusion.

[0145] It should be noted that this specification does not limit the fusion order. For example, the time domain filtering system 100 can also first fuse the first reference image block and the upper reference image block to obtain an intermediate fused image block, and then fuse the intermediate fused image block and the left reference image block to obtain the fused image block.

[0146] Method 2

[0147] In some embodiments, Fig.10 As shown, the time domain filtering system 100 fuses the first reference image block and the M second reference image blocks to obtain a fused image block, and selects the one with the smallest difference from image block #1 from the fused image block and the first reference image block as the prediction block. If the image content difference between the fused image block and image block #1 is smaller, the time domain filtering system 100 uses the fused image block as the prediction block; if the image content difference between the first reference image block and image block #1 is smaller, the time domain filtering system 100 uses the first reference image block as the prediction block.

[0148] In the method 2, the execution steps of the time domain filtering system 100 to obtain the fused image block can refer to the relevant description in the method 1, which will not be repeated here.

[0149] In this embodiment, while considering reducing the block effect in the filtering process, the efficiency of subsequent video encoding is taken into account. If the difference between the fused image block and the first reference image block is too large, the first reference image block is selected in the filtering process to reduce the amount of information in the video encoding and improve the encoding efficiency.

[0150] Through the above-mentioned method 1 or method 2, the time domain filtering system 100 obtains the prediction block corresponding to the image block #1 with a reference frame as a reference. When the number of reference frames is n, n is an integer greater than 1. The time domain filtering system 100 can obtain the prediction blocks with other reference frames as references in a similar manner, and a total of n prediction blocks can be obtained.

[0151] S340: Filter the image block #1 based on the prediction block to obtain a filtering result of the image block #1.

[0152] Exemplarily, the time domain filtering system 100 may filter based on the following formula:

[0153]

[0154] Where, f represents the pixel of the filtered image block #1, P0 represents the original pixel of the image block #1, and P k represents the pixel of the kth prediction block among the n prediction blocks, ω k represents the filtering weight corresponding to the kth prediction block, and n represents the number of prediction blocks involved in filtering.

[0155] ω k Determined based on the following formula:

[0156]

[0157] Among them, D kRepresents the difference between image block #1 and the k-th prediction block, which can be determined by the Euclidean distance / square error sum of the pixels of image block #1 and the k-th prediction block; h is the key parameter for controlling the weight attenuation in the Euclidean distance measurement process.

[0158] D k Determined based on the following formula:

[0159]

[0160] Where Ω represents the pixel set of the image block or prediction block, P0(z) represents the zth original pixel in the pixel set of image block #1, and P k (z) represents the zth pixel in the pixel set of the kth prediction block.

[0161] h is determined by the original noise in image block #1, which can adapt the filter coefficient to the image content with different noises and can be determined based on the following formula:

[0162] h=t1(t2+ln(σ+t3))

[0163] Among them, σ is the standard deviation of the original noise, h and σ are in a monotonically increasing relationship, and through experiments and experience, it can be determined that t1=4, t2=0.7, and t3=0.5.

[0164] The time domain filtering system 100 may filter other image blocks except the image block #1 corresponding to the current frame in a similar manner, thereby obtaining the multiple filtered image blocks.

[0165] S350: Determine a filtering result of the current frame based on the filtering results of each image block in the current frame.

[0166] The time domain filtering system 100 may concatenate the filtering results corresponding to the multiple image blocks to obtain the filtering result of the current frame, and may filter other image frames in the video in a similar manner, thereby finally obtaining a filtered video.

[0167] In summary, the time domain filtering method and system provided in this specification is based on the idea that the motion information of adjacent image blocks in a video image frame is relatively close. In the process of motion compensation, the motion information of adjacent image blocks is used to help improve the accuracy of motion compensation. Therefore, when filtering, it is possible to remove noise in the video, and to perform pixel interpolation on the reference blocks obtained by motion compensation based on the motion information of the current image block and the adjacent image blocks, smooth the image content in the prediction block, reduce the block effect of the filtered video, improve the quality of the filtered video, and improve the encoding efficiency of the video in the subsequent video compression process.

[0168] On the other hand, the present specification provides a computer-readable non-transitory storage medium, storing at least one set of executable instructions for performing time-domain filtering. When the executable instructions are executed by the processor, the executable instructions guide the processor to implement the steps of the method P300 described in the present specification. In some possible implementations, various aspects of the present specification can also be implemented in the form of a program product, which includes a program code. When the program product is run on the time-domain filtering system 100, the program code is used to make the time-domain filtering system 100 perform the steps of the method P300 described in the present specification. The program product for implementing the above method can use a portable compact disk read-only memory (CD-ROM) to include program code and can be run on the time-domain filtering system 100. However, the program product of the present specification is not limited to this. In the present specification, the readable storage medium can be any tangible medium containing or storing a program, which can be used by the instruction execution system or used in combination with it. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media include: an electrical connection with one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. Program code for performing the operations of the present specification may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the time-domain filtering system 100, partially on the time-domain filtering system 100, as a stand-alone software package, partially on the time-domain filtering system 100 and partially on a remote computing device, or entirely on a remote computing device.

[0169] The term "and / or" in the embodiments of this specification describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0170] The terms "first", "second" and the like in this specification are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit a specific order or sequence.

[0171] The term “plurality” mentioned in this specification shall mean two or more than two unless otherwise stated.

[0172] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require a specific order or a continuous order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0173] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented only by way of example and may not be limiting. Although not explicitly stated herein, those skilled in the art will appreciate that this specification requires various reasonable changes, improvements and modifications to the embodiments. These changes, improvements and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0174] In addition, certain terms in this specification have been used to describe embodiments of this specification. For example, "one embodiment", "an embodiment" and / or "some embodiments" mean that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of this specification. Therefore, it can be emphasized and should be understood that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics may be appropriately combined in one or more embodiments of this specification.

[0175] It should be understood that in the foregoing description of the embodiments of this specification, in order to help understand a feature and for the purpose of simplifying this specification, this specification combines various features in a single embodiment, figure or its description. However, this does not mean that the combination of these features is necessary. When reading this specification, it is entirely possible for a person skilled in the art to mark out some of the devices as separate embodiments. In other words, the embodiments in this specification can also be understood as the integration of multiple secondary embodiments. This is also true when the content of each secondary embodiment is less than all the features of a single aforementioned disclosed embodiment.

[0176] Each patent, patent application, publication of patent application, and other materials, such as articles, books, specifications, publications, documents, documents, etc., cited in this disclosure (excluding any historical review documents related thereto) are hereby incorporated by reference for all purposes related to this disclosure, such as in the specification and claims of this disclosure. However, if there is any inconsistency or conflict between the descriptions, definitions, and / or terminology of the above materials and the descriptions, definitions, and / or terminology used in this disclosure, the descriptions, definitions, and / or terminology used in this disclosure shall prevail.

[0177] Finally, it should be understood that the embodiments of the application disclosed herein are explanations of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are only used as examples and not as limitations. Those skilled in the art can adopt alternative configurations according to the embodiments in this specification to implement the applications in this specification. Therefore, the embodiments of this specification are not limited to the embodiments accurately described in the application.

Claims

1. A time domain filtering method for performing time domain filtering on each image frame in a video, comprising: Obtaining a current frame and a reference frame from the video, wherein the current frame is divided into a plurality of image blocks; For each image block in the current frame, taking the reference frame as a reference, determining motion information of the image block and motion information of N adjacent blocks of the image block, wherein the motion information of the image block represents a position offset of content of the image block between the current frame and the reference frame, and the motion information of each adjacent block represents a position offset of content of the adjacent block between the current frame and the reference frame, and N is an integer greater than or equal to 1, determining a prediction block corresponding to the image block based on the motion information of the image block, the motion information of the N adjacent blocks and the reference frame, The image block is filtered based on the prediction block to obtain a filtering result of the image block; and a filtering result of the current frame is determined based on the filtering results of each image block in the current frame.

2. The method according to claim 1, wherein: The determining, based on the motion information of the image block, the motion information of the N adjacent blocks and the reference frame, a prediction block corresponding to the image block comprises: Determining, in the reference frame, a first reference image block that matches content of the image block based on motion information of the image block and a position of the image block in the current frame; Determine M target adjacent blocks from the N adjacent blocks, and for each target adjacent block, determine a second reference image block in the reference frame that matches the content of the target adjacent block based on motion information of the target adjacent block and a position of the image block in the current frame, to obtain a total of M second reference image blocks, where M is an integer less than or equal to N; and A prediction block corresponding to the image block is determined based on the first reference image block and the M second reference image blocks.

3. The method according to claim 2, wherein: The determining, based on the first reference image block and the M second reference image blocks, a prediction block corresponding to the image block comprises: The first reference image block and the M second reference image blocks are fused to obtain a fused image block, and the fused image block is used as the prediction block.

4. The method according to claim 2, wherein: The determining, based on the first reference image block and the M second reference image blocks, a prediction block corresponding to the image block comprises: Fusing the first reference image block and the M second reference image blocks to obtain a fused image block; and Among the fused image block and the first reference image block, the one with the smallest difference from the image block is selected as the prediction block.

5. The method according to claim 3 or 4, wherein: The M target adjacent blocks include a left adjacent block and an upper adjacent block, and the M second reference image blocks include: a left reference image block determined for the left adjacent block and an upper reference image block determined for the upper adjacent block, The step of fusing the first reference image block and the M second reference image blocks to obtain a fused image block includes: Fusing the first reference image block and the left reference image block to obtain an intermediate fused image block; and The intermediate fused image block and the upper reference image block are fused to obtain the fused image block.

6. The method according to claim 5, wherein fusing the first reference image block and the left reference image block to obtain an intermediate fused image block comprises: The left half area of ​​the left reference image block is weightedly fused to the left half area of ​​the first reference image block according to pixel columns to obtain the intermediate fused image block, wherein the total number of pixel columns of the left reference image block or the first reference image block is W, the weight corresponding to the i-th column of pixels in the left half area of ​​the left reference image block is positively correlated with i, the weight corresponding to the i-th column of pixels in the left half area of ​​the first reference image block is negatively correlated with i, and i is less than or equal to W / 2.

7. The method according to claim 6, wherein: The intermediate fused image block is obtained based on the following formula: Among them, P′(i,j) represents the pixel in the jth row and i-th column in the intermediate fused image block, P(i,j) represents the pixel in the jth row and i-th column in the first reference image block, w(i) represents the weight corresponding to the pixel in the i-th column in the left reference image block, LP(i,j) represents the pixel in the jth row and i-th column in the left reference image block, and a is a preset value.

8. The method according to claim 5, wherein fusing the intermediate fused image block with the upper reference image block to obtain the fused image block comprises: The upper half area of ​​the upper reference image block is weightedly fused to the upper half area of ​​the intermediate fused image block according to pixel rows to obtain the fused image block, wherein the total number of pixel rows of the upper reference image block or the intermediate fused image block is H, the weight corresponding to the j-th row of pixels in the upper half area of ​​the upper reference image block is positively correlated with j, the weight corresponding to the j-th row of pixels in the upper half area of ​​the intermediate fused image block is negatively correlated with j, and j is less than or equal to H / 2.

9. The method according to claim 8, wherein: The fused image block is obtained based on the following formula: Among them, P″(i,j) represents the pixel in the jth row and i-th column of the fused image block, P′(i,j) represents the pixel in the jth row and i-th column of the intermediate fused image block, w(j) represents the weight corresponding to the pixel in the jth row of the upper reference image block, TP(i,j) represents the pixel in the jth row and i-th column of the upper reference image block, and b is a preset value.

10. The method according to claim 2, wherein: The determining of M target adjacent blocks from the N adjacent blocks includes: For each neighboring block among the N neighboring blocks, if the difference between the motion information of the neighboring block and the motion information of the image block is within a preset range, the neighboring block is determined as the target neighboring block.

11. The method according to claim 10, wherein: The difference between the motion information of the adjacent block and the motion information of the image block is within a preset range, including at least one of the following: The difference between the horizontal component information in the motion information of the adjacent block and the horizontal component information in the motion information of the image block is within a predetermined range; or The difference between the vertical component information in the motion information of the adjacent block and the vertical component information in the motion information of the image block is within a second preset range.

12. The method according to claim 1, wherein: The method further comprises: Performing hierarchical motion estimation on the current frame and the reference frame to obtain a motion estimation result, wherein the motion estimation result includes position information of each image block in the current frame in the current frame and motion information of each image block; The determining the motion information of the image block and the motion information of N adjacent blocks of the image block comprises: Based on the position information of the image block in the current frame, query the motion information of the image block from the motion estimation result, and For each adjacent block, based on the position information of the image block in the current frame and the positional relationship between the image block and the adjacent block, the position information of the adjacent block in the current frame is determined, and based on the position information of the adjacent block in the current frame, the motion information of the adjacent block is queried from the motion estimation result.

13. The method according to claim 1, wherein: The N adjacent blocks include at least one of an upper adjacent block, a lower adjacent block, a left adjacent block, and a right adjacent block.

14. A time domain filtering system, comprising: at least one storage medium storing at least one instruction set; as well as At least one processor is communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set when running, and implements the method as described in any one of claims 1-13 according to the instructions of the at least one instruction set.

Images