Image filtering processing method, device, equipment, medium and program product

Abstract

Embodiments of the present specification provide an image filtering processing method, device, equipment, medium and program product, wherein the image filtering processing method comprises: screening a candidate affine motion region in a target image, and calculating a translation processing cost of the candidate affine motion region; obtaining affine motion information of the candidate affine motion region, and calculating an affine processing cost based on the affine motion information; in the case that the translation processing cost and the affine processing cost satisfy an affine strategy, determining the affine motion information as target motion information of the candidate affine motion region; performing filtering processing on the target motion information, the candidate affine motion region and a reference image block corresponding to the candidate affine motion region to obtain an affine filtering sub-block, and obtaining a filtering processing image based on each affine filtering sub-block and each translation filtering sub-block corresponding to the target image. When the affine strategy is satisfied, the affine motion information is used for subsequent filtering processing, and the robustness of image filtering in a complex motion scene is enhanced.

Description

Technical Field

[0001] This specification relates to the field of video encoding and decoding technology, and in particular to an image filtering processing method, apparatus, device, medium, and program product. Background Technology

[0002] In real-time video services such as short videos and live streaming, it is urgent to significantly accelerate video encoding speed while ensuring compression efficiency in order to improve user experience.

[0003] While existing coding standards acquire rich motion information during the precoding stage, this information is not effectively reused in subsequent processing modules such as MCTF, leading to redundant searches and wasted computational resources. Especially under the pyramid search architecture, low-precision initial searches are prone to error accumulation, making it difficult to accurately model complex motions such as rotation and scaling. This results in inaccurate motion compensation and biased filtering weights, thereby reducing the effectiveness of temporal filtering and the overall coding quality.

[0004] Therefore, there is an urgent need for an image filtering method that can improve the accuracy of motion estimation for affine motion. Summary of the Invention

[0005] In view of this, embodiments of this specification provide an image filtering processing method. One or more embodiments of this specification also relate to an image filtering processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, an image filtering processing method is provided, comprising: Candidate affine motion regions are selected in the target image, and the translation processing cost of the candidate affine motion regions is calculated. Obtain affine motion information of candidate affine motion regions and calculate affine processing cost based on affine motion information; When the translation processing cost and the affine processing cost satisfy the affine strategy, the affine motion information is determined as the target motion information of the candidate affine motion region. Based on the target motion information, candidate affine motion regions and corresponding reference image blocks, filtering is performed to obtain affine filter sub-blocks. Based on each affine filter sub-block and each translation filter sub-block corresponding to the target image, a filtered image is obtained.

[0007] According to a second aspect of the embodiments of this specification, an image filtering processing apparatus is provided, comprising: The filtering module is configured to filter candidate affine motion regions in the target image and calculate the translation processing cost of the candidate affine motion regions. The calculation module is configured to acquire affine motion information of candidate affine motion regions and calculate the affine processing cost based on the affine motion information. The determination module is configured to determine the affine motion information as the target motion information of the candidate affine motion region when the translation processing cost and the affine processing cost satisfy the affine policy. The filtering module is configured to perform filtering based on target motion information, candidate affine motion regions and corresponding reference image blocks to obtain affine filter sub-blocks, and obtain the filtered image based on each affine filter sub-block and each translation filter sub-block corresponding to the target image.

[0008] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above-described image filtering processing method.

[0009] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described image filtering processing method.

[0010] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described image filtering processing method.

[0011] According to a sixth aspect of the embodiments of this specification, a method for storing a bitstream is provided, comprising storing the bitstream in a storage medium, the bitstream being generated by the image filtering processing method of the first aspect.

[0012] According to a seventh aspect of the embodiments of this specification, a method for transmitting a bit stream is provided, including transmitting the bit stream, the bit stream being generated by the image filtering processing method of the first aspect.

[0013] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a bit stream thereon, the bit stream being generated by the image filtering processing method of the first aspect.

[0014] One embodiment of this specification implements the following steps: screening candidate affine motion regions in a target image and calculating the translation processing cost of the candidate affine motion regions; obtaining affine motion information of the candidate affine motion regions and calculating the affine processing cost based on the affine motion information; determining the affine motion information as the target motion information of the candidate affine motion regions when the translation processing cost and the affine processing cost satisfy the affine policy; performing filtering processing based on the target motion information, the candidate affine motion regions, and the reference image blocks corresponding to the candidate affine motion regions to obtain affine filter sub-blocks; and obtaining the filtered image based on each affine filter sub-block and each translation filter sub-block corresponding to the target image.

[0015] During the MCTF process, for regions with complex motion, affine motion information is introduced and compared with its translation processing cost. When the affine strategy is satisfied, the affine model is used first, thereby more accurately depicting non-translational motion (such as rotation, scaling, and perspective distortion). Based on this, the generated affine filter sub-blocks can better align with the reference image blocks, effectively improving motion compensation accuracy, reducing filtering artifacts caused by motion model mismatch, and enhancing the robustness and reconstruction quality of image filtering in complex motion scenes. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of an example of an MCTF filter; Figure 2 This is a typical diagram of a pyramid search structure; Figure 3 This is a flowchart of an image filtering processing method provided in one embodiment of this specification; Figure 4 This is a flowchart illustrating the processing procedure of an image filtering method provided in one embodiment of this specification. Figure 5 This is a schematic diagram of the structure of an image filtering processing device provided in one embodiment of this specification; Figure 6 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0017] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0018] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0019] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0020] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0021] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0022] GOP stands for Group of Pictures, which is a set of images. A GOP consists of a series of consecutive video frames, where the first frame is a keyframe (an intra-predictive frame, I-frame, or a forward predictive frame, P-frame), and subsequent frames are bidirectional predictive frames (B-frames).

[0023] Lookahead: Pre-coding, an initial step in the coding pipeline, involves analyzing the input video sequence before formal encoding to understand its motion characteristics and complexity. This provides guidance for subsequent coding decisions, such as bitrate control and motion estimation.

[0024] MCTF stands for Motion Compensated Temporal Filtering. It is a standardized temporal filtering method based on motion compensation that enhances the compression performance and improves the quality of video sequences by estimating and compensating for motion between frames.

[0025] MV stands for Motion Vector. In video coding, motion vectors represent the movement of coded blocks from one frame to another. Vector values ​​typically include information in both the horizontal and vertical directions, representing the displacement of the block along the time axis within a frame.

[0026] In short video scenarios, videos uploaded by creators need to be transcoded by the server before being distributed to users. To enable users to watch creators' videos as quickly as possible, the platform aims to encode the video as fast as possible while maintaining compression efficiency. In live streaming scenarios, to reduce buffering and ensure smoother streaming, the encoder needs to encode the captured video at the fastest possible speed. Therefore, the platform needs to improve the encoding speed of the video encoder. Existing encoding standards (such as H.266) obtain rich motion information in their pre-encoding stage, but this information is not fully utilized in subsequent processing modules such as MCTF, leading to a waste of computational resources during large-scale searches. Furthermore, in the implementation of MCTF, due to the multi-level complexity of the pyramid search process and the uncertainty of inter-frame motion, low-precision searches, while reducing computational load, are also prone to accumulating search errors, thus affecting the encoding efficiency and quality of the video.

[0027] Motion-compensated temporal filtering (MCTF) is a technique used in video coding systems to enhance the temporal signal processing of a video sequence by utilizing motion information between frames. Its main purpose is to reduce temporal redundancy in video, improve video compression efficiency, and simultaneously maintain or even enhance video quality.

[0028] See Figure 1 As shown, Figure 1 This is a schematic diagram of an example of an MCTF filter. Figure 1 The complete processing flow from precoding information input to encoding result output is presented: Precoding information first generates filtered reference frames, which are then downsampled sequentially by 2x and 4x reference frames, with each level performing coded block division and integer pixel motion estimation; subsequently, motion search is refined step-by-step at 1 / 2, 1 / 4, and 1 / 8 pixel precision; all of the above processes are input to the temporal filter block, which calculates the filter strength based on four inputs: reference block acquisition, block prediction cost, reference frame distance, and reference frame weights, and then outputs the final encoding result through weighted fusion. More specifically, MCTF mainly achieves motion compensation and temporal filtering through the following steps: Motion estimation: The motion trajectory of each pixel or block between adjacent frames is determined using a fixed-size block matching algorithm and search modes such as Full Search, Three-Step Search, and Pyramid Search, and recorded as motion vectors in the MCTF. In common encoders, to balance search accuracy and computational complexity, a pyramid structure built with different downsampling rates is typically used for the search, expanding the search from high downsampling rates to high accuracy.

[0029] See Figure 2 As shown, Figure 2 This is a schematic diagram of a typical pyramid search structure. Figure 2 The structure is divided into three layers: the bottom layer L0, the middle layer L1, and the top layer L2. These three layers are arranged in a bottom-up, progressively upsampled (or downsampled) relationship, forming a pyramid structure with decreasing spatial resolution. L0 is the highest resolution layer, L1 is the intermediate resolution layer obtained by downsampling, and L2 is the lowest resolution layer. This structure is used for a coarse-fine search strategy in motion estimation: a fast coarse search is first performed in L2 to obtain the initial motion vector, and then the search is progressively passed to L1 and L0 for refinement, thereby significantly reducing computational complexity while ensuring search accuracy.

[0030] Motion compensation: Adjusts the pixel values ​​of the current frame based on motion information, using motion vectors to locate and compensate for pixel positions within the frame, aligning homogeneous blocks in the current frame and the reference frame. Temporal filtering: Weights are calculated for homogeneous blocks in different frames using a series of similarity metrics. Bidirectional or unidirectional filtering is then used to filter motion-compensated frames to remove short-term fluctuations and noise, thereby improving the visual quality of the video.

[0031] MCTF effectively removes redundant information in the temporal domain, improves the reference efficiency of temporal information during encoding, significantly enhances video compression efficiency, and achieves higher-quality video transmission within a given bandwidth. By compensating for and filtering noise introduced by motion, MCTF technology ensures smooth video transitions and reduces blockiness and glitches. It can adapt to video content at various resolutions and frame rates and is widely used in live video applications.

[0032] Conventional MCTF temporal filtering techniques achieve temporal denoising and reference frame enhancement through block translation motion estimation / compensation. However, this significantly increases alignment errors when there are complex motions such as rotation, scaling, perspective changes, and camera shake. This can lead to inaccurate filtering weights, ghosting / ghosting after filtering, or the need to reduce the filtering intensity, thereby reducing the denoising benefits.

[0033] To address the aforementioned problems, this specification provides an image filtering processing method. This specification also relates to an image filtering processing apparatus, a computing device, a computer-readable storage medium, and a program product, which will be described in detail in the following embodiments.

[0034] See Figure 3 , Figure 3 This is a flowchart of an image filtering processing method provided in one embodiment of this specification, specifically including the following steps 302-308.

[0035] Step 302: Filter candidate affine motion regions in the target image and calculate the translation processing cost of the candidate affine motion regions.

[0036] Candidate affine motion regions refer to local image patches in a target image that may exhibit non-translational motion, typically characterized by inconsistencies in the motion vector field or regions with high residual energy. For example, in a video frame containing a person turning around or a camera zooming in, the arm region or background edge region can serve as candidate affine motion regions.

[0037] Translation processing cost refers to the rate distortion cost or predicted residual energy introduced by motion estimation and compensation under the assumption that only translational motion occurs in the region. For example, the SAD (sum of absolute errors) or SATD (sum of absolute errors after Hadamard transform) value calculated after matching the reference block with the translational motion vector is a quantitative form of translation processing cost.

[0038] In practical applications, the system first divides the target image into multiple image blocks to be processed. Then, based on indicators such as motion vector consistency, residual distribution characteristics, or gradient changes, it identifies image blocks with complex motion behavior as candidate affine motion regions. Next, the system uses existing motion estimation results or motion information from neighboring blocks to determine a translational motion vector for each candidate affine motion region, and extracts the corresponding reference image block from the reference image based on this vector. Furthermore, the system calculates the prediction error between the candidate region and its reference image block, and combines this with the coding bit overhead to form the translation processing cost. The calculation of the translation processing cost can be based on rate-distortion optimization criteria or solely on residual energy metrics; the specific implementation can be flexibly configured according to system resources and accuracy requirements.

[0039] For example, in a video sequence containing a vehicle driving by with slight camera rotation, the system divides the current frame (target image) into 8×8 or 16×16 blocks. When processing the wheel area, the system detects that the motion vectors within this block are radially distributed, significantly different from the motion directions of the surrounding vehicle blocks, and therefore marks it as a candidate affine motion region. Subsequently, the system obtains the corresponding position of this region in the previous frame and uses the translational motion vectors of neighboring blocks (e.g., horizontally moving 5 pixels to the right) to extract a reference block. Then, the system calculates the sum of the absolute values ​​of the pixel differences between the candidate region and the reference block, obtaining a translation processing cost of 850. This cost will be used for subsequent comparison with the affine processing cost to determine whether to enable the affine motion model. The entire process completes the initial identification and cost evaluation of complex motion regions without introducing additional motion search.

[0040] Further, the target image is the current frame; filtering candidate affine motion regions in the target image includes: dividing the current frame into multiple filtering units for filtering processing; for the first filtering unit, obtaining first motion information of the corresponding position of the first filtering unit in adjacent frames and the preset surrounding range of the corresponding position; obtaining second motion information of the first filtering unit in adjacent filtering units in the current frame, wherein the adjacent frame is a reference frame that is adjacent to the current frame in the time sequence, and the first filtering unit is any filtering unit in the current frame; if affine motion information exists in the first motion information or the second motion information, the first filtering unit is determined as a candidate affine motion region.

[0041] The current frame is the target image to be filtered, and it is located at the position to be processed in the time series, such as the nth frame in a video sequence.

[0042] A filtering unit is a basic image block unit in the current frame used to perform an independent filtering task. Its size can be 8×8, 16×16 or 32×32 pixels, covering the entire frame image.

[0043] Adjacent frames are reference frames that are directly adjacent to the current frame in time, including forward reference frames (such as the (n-1)th frame) or backward reference frames (such as the (n+1)th frame), used to provide spatiotemporal motion cues.

[0044] The first motion information is the set of motion parameters associated with the corresponding position of the first filtering unit in adjacent frames and its preset surrounding range, which can be used to determine whether affine motion behavior exists.

[0045] The second motion information is the final motion parameters of the first filtering unit corresponding to the spatially adjacent filtering units in the current frame, reflecting the motion consistency of the local area.

[0046] Affine motion information consists of motion parameters expressed in the form of affine transformations, typically composed of a 2×3 matrix or six degrees of freedom parameters, used to describe non-translational spatial deformations.

[0047] In practical applications, the system first divides the current frame into multiple filtering units. For any first filtering unit, the system locates its corresponding position in adjacent frames and extends this location to a preset surrounding area. This area can be defined as an M×N pixel region centered on the corresponding position or a neighborhood composed of several filtering units. Within this area, the system retrieves all encoded or estimated motion information to determine whether affine motion information exists. Simultaneously, the system acquires the motion information of filtering units spatially adjacent to the first filtering unit in the current frame. Typical adjacent directions include left, top, and upper left. If either the first or second motion information contains affine motion information, the system identifies the first filtering unit as a candidate affine motion region.

[0048] For the step "obtaining the first motion information of the corresponding position of the first filtering unit in adjacent frames and the first motion information within the preset surrounding range of the corresponding position", one possible approach is to construct a search window of fixed size with the same coordinate position of the first filtering unit in adjacent frames as the center, and extract the motion information of all encoded blocks within the window; another possible approach is to dynamically adjust the size of the surrounding range according to the time distance, with the range being smaller the closer the distance and larger the search area the opposite.

[0049] For the step "obtain the second motion information of the first filtering unit's neighboring filtering units in the current frame", one option is to check only the neighboring filtering units in the left, top, and top-left directions; another option is to extend to neighboring units in more directions, such as the bottom right, to enhance the robustness of spatial continuity judgment.

[0050] In the embodiments of this specification, by jointly analyzing the types of motion information in the spatiotemporal neighborhood, potential affine motion regions can be effectively identified without increasing the global computational burden, providing reliable input for subsequent differentiated filtering strategies.

[0051] For example, in a video sequence containing a vehicle turning scene, the current frame is frame 15, which the system divides into 16×16 filtering units. For a first filtering unit located in the front wheel area of ​​the vehicle, the system first locates its corresponding position in frame 14 and expands a 32×32 pixel perimeter around that position. Within this perimeter, the system finds that two encoded blocks in frame 14 have used a 2×3 affine transformation matrix as motion parameters; therefore, the first motion information contains affine motion information. Simultaneously, the system examines the adjacent filtering units to the left and above this first filtering unit in the current frame and finds that the final motion information of the left unit is also affine. Since affine motion information exists in both the first and second motion information, the system identifies this first filtering unit as a candidate affine motion region. The entire process relies only on existing motion information, eliminating the need to re-perform motion estimation, thus achieving efficient filtering.

[0052] Furthermore, the target image is the current frame, and the current frame has an image pyramid containing multiple resolution levels; the process of filtering candidate affine motion regions in the target image includes: dividing the current frame into multiple filtering units for filtering processing; for the first filtering unit of the target resolution level, detecting whether there is a downsampling level at the target resolution level; if there is a downsampling level at the target resolution level, and the downsampling level satisfies the affine activation strategy, then the region to which the first filtering unit belongs is marked as a candidate affine motion region.

[0053] The current frame is the target image to be filtered, and it is in the processing position in the time series, such as the nth frame in the video encoding or post-processing process.

[0054] An image pyramid is a collection of multi-resolution images generated by downsampling the current frame level by level. It contains multiple levels, usually starting from the original resolution (L0) and generating lower resolution levels such as L1, L2, etc. The image size of each level is about half that of the previous level.

[0055] The filtering unit is the basic image block unit in the current frame used to perform independent filtering tasks, and it has a corresponding spatial position relationship in different pyramid levels.

[0056] The target resolution level is the pyramid level currently being processed, such as L0 or L1, at which the system performs candidate region filtering for the filtering units.

[0057] The downsampling level is the level in the image pyramid that is one level lower than the target resolution level, that is, the adjacent level with smaller resolution. For example, when the target level is L0, the downsampling level is L1.

[0058] The affine activation strategy is a rule for determining whether to mark the current region as a candidate affine motion region. It is based on motion information from the downsampling level, including conditions such as motion type, cost difference, or spatial consistency.

[0059] In practical applications, the system first constructs an image pyramid for the current frame, containing multiple resolution levels. Then, the current frame is divided into multiple filtering units, and for any first filtering unit in the target resolution level, it checks whether a downsampling level exists. If so, the system obtains the motion information and related processing cost of the corresponding position of the first filtering unit in the downsampling level, and makes a judgment based on a preset affine activation strategy. The affine activation strategy can be based on any of the following conditions: the final motion vector at the corresponding position in the downsampling level is in affine transformation form; the translation processing cost at the corresponding position in the downsampling level exceeds a set threshold; or the position of the current filtering unit in the downsampling level is not the first sub-block in the upper left corner, and the difference between its motion vector and the motion vector of the upper left sub-block is greater than a preset threshold k. If any one of these conditions is met, the system marks the region to which the first filtering unit belongs as a candidate affine motion region.

[0060] For the step "If a downsampling level exists at the target resolution level and the downsampling level satisfies the affine activation strategy, then mark the region to which the first filtering unit belongs as a candidate affine motion region", one possible approach is to use only the motion vector type in the downsampling level as the judgment criterion, that is, as long as it is affine, the marking is triggered; another possible implementation is to comprehensively consider motion cost and vector difference, and set combined conditions, such as requiring translation cost to exceed the standard and vector difference to be greater than k at the same time, in order to improve the screening accuracy.

[0061] In the embodiments of this specification, by utilizing the motion information transmission mechanism between the levels of the image pyramid, potential affine regions can be quickly identified at higher levels, avoiding high-overhead full-area affine detection at the original resolution, thereby improving screening efficiency and robustness.

[0062] For example, in a video sequence containing zoom-out shots, the current frame is frame 20. The system constructs a three-level image pyramid for it: L0 (original resolution), L1 (half resolution), and L2 (quarter resolution). When processing a first filter unit located at the edge of a building at the L0 level, the system detects the existence of a downsampling level L1. Subsequently, the system locates the region corresponding to this filter unit in the L1 level and finds that the final motion vector at this location is a 2×3 affine transformation matrix, which meets condition 1.2.3 in the affine transformation activation strategy. Therefore, the system marks the region to which the first filter unit belongs as a candidate affine motion region. Another filter unit located in a grass texture region has a translation processing cost of 850 at its corresponding position in the L1 level, exceeding the preset threshold of 800, satisfying condition 1.2.4, and is also marked as a candidate region. There is also a filtering unit located near the center of the rotating windmill blades. In the L1 level, its corresponding position is not the top-left sub-block, and the L1 norm of the difference between its motion vector and the top-left sub-block's vector is 12, which is greater than the preset threshold k=10, satisfying condition 1.2.5, and it is also marked. The entire process is based entirely on the existing motion decisions between the pyramid levels, without needing to re-perform complex estimations at the L0 level, thus achieving efficient and multi-level affine region recognition.

[0063] Furthermore, the target image is the current frame, and the current frame corresponds to a time-domain adjacent frame; the translation processing cost of the candidate affine motion region is calculated, including: performing motion search on the candidate affine motion region based on the translation motion model to obtain translation motion information; extracting reference image blocks in the adjacent frames according to the translation motion information, and calculating the pixel difference between the candidate affine motion region and the reference image block to obtain the translation processing cost.

[0064] Translational motion models are a way of representing motion that includes only two degrees of freedom: horizontal and vertical displacement. They are usually described in the form of two-dimensional motion vectors (MV).

[0065] Translational motion information is the optimal motion vector obtained after performing motion search on the candidate region based on the translational motion model, which is used to indicate the best matching position in adjacent frames.

[0066] The reference image patch is an image patch extracted from adjacent frames based on translational motion information and spatially aligned with the candidate affine motion region, used for difference calculation with the current region.

[0067] Translation processing cost is a pixel-level measure of the difference between the candidate affine motion region and its corresponding reference image block, usually expressed by metrics such as SAD (sum of absolute errors), SSD (sum of squared errors), or SATD (sum of absolute errors after Hadamard transform).

[0068] In practical applications, for each candidate affine motion region, the system first performs a motion search within a preset search range of its adjacent frames based on a translational motion model. This search process aims to find the displacement vector that minimizes the pixel difference between the current region and a certain image patch in an adjacent frame, i.e., the translational motion information. The search range can be a fixed window (e.g., ±32 pixels) or dynamically adjusted according to the scene. After obtaining the translational motion information, the system locates and extracts the corresponding reference image patch in the adjacent frames based on this vector, ensuring that its spatial size is consistent with the candidate region. Subsequently, the system calculates the difference between the candidate region and the reference image patch pixel by pixel and accumulates them according to a preset criterion (e.g., SAD) to obtain the translation processing cost of the candidate region. This cost reflects the degree of prediction inaccuracy when using only the translation model; a higher value indicates that the translation model is less able to accurately describe the motion characteristics of the region.

[0069] For the step "to perform motion search on candidate affine motion regions based on translational motion model and obtain translational motion information", one possible approach is to adopt a full search strategy and traverse all possible displacements within the entire search window; another possible approach is to adopt fast algorithms such as hierarchical search or diamond search to reduce computational complexity.

[0070] For the step "Calculate the pixel difference between the candidate affine motion region and the reference image block", one option is to use SAD as the cost function, which is suitable for integer pixel precision; another option is to use SATD after subpixel interpolation to more accurately reflect the frequency domain energy difference.

[0071] In the embodiments of this specification, by quantifying the reconstruction error of the translation model on the candidate region, an objective basis is provided for the subsequent affine model activation decision, avoiding the misactivation of high-complexity processing for regions that can be fully modeled by translation.

[0072] For example, in a video containing a pedestrian walking against a static background, the current frame is frame 30. The system has identified a candidate affine motion region located in the area where the pedestrian's arm is swinging. For this region, the system sets a search window of ±24 pixels centered at the same coordinates in frame 29 and performs a translational motion search based on SAD. After traversal comparison, the system finds the optimal translational motion vector to be (5, -3), i.e., 5 pixels to the right and 3 pixels upward. Based on this, the system extracts an 8×8 reference image block at the corresponding position from frame 29. Subsequently, the system subtracts the absolute value of each pixel in the candidate region in the current frame from the corresponding pixel in the reference image block and accumulates them to obtain a SAD value of 620, which serves as the translation processing cost. Since this value is significantly higher than a preset threshold (e.g., 400), the system determines that the translation model is insufficient to accurately describe the motion of this region and that the affine processing flow must be initiated. The entire process only involves conventional motion search and pixel difference calculation, without introducing additional affine operations, effectively controlling the initial judgment overhead.

[0073] During video filtering or encoding, some regions may be initially identified as affine motion regions. However, if their texture is too flat or lacks structural features, using an affine model will not only fail to improve reconstruction quality but will also introduce redundant computation. Therefore, a further step is taken to screen candidate affine motion regions in the target image, including: screening initial affine motion regions in the target image; calculating the texture complexity of the initial affine motion regions; comparing the texture complexity with a corresponding effective threshold; and determining the initial affine motion region as a candidate affine motion region if the texture complexity is greater than or equal to the effective threshold.

[0074] The initial affine motion region is a local image region that has been preliminarily identified as potentially having affine motion in the previous steps, such as a filter unit marked based on spatiotemporal motion information or pyramid hierarchy decision.

[0075] Texture complexity is an indicator used to quantify the richness of structure in a local region of an image. It reflects whether the region has the visual details required to support affine motion modeling and is usually measured by statistics such as gradient magnitude, pixel variance, or high-frequency energy.

[0076] The effective threshold is a preset decision boundary value used to compare with the texture complexity and determine whether to retain the region for subsequent affine processing. Different texture metric dimensions can correspond to different effective thresholds.

[0077] In practical applications, the system first acquires a set of regions in the target image that have been marked as initial affine motion regions. For each initial affine motion region, the system calculates its texture complexity. Specifically, the system can calculate the sum of the absolute values ​​of the horizontal gradients and the absolute values ​​of the vertical gradients of all pixels within the region, as well as the variance of the pixel grayscale values. These statistics together constitute a comprehensive evaluation of the texture richness of the region. Subsequently, the system compares each texture complexity indicator with its corresponding effective threshold. If any one of the three items—horizontal gradient, vertical gradient, and variance—is less than its effective threshold, the texture of the region is considered too simple to support an effective estimation of the affine transformation, and the system will skip the subsequent affine processing of that region; conversely, only when all relevant texture complexities are greater than or equal to their respective effective thresholds will the system formally identify the initial affine motion region as a candidate affine motion region.

[0078] For the step "Calculate the texture complexity of the initial affine motion region", one option is to use only pixel variance as a single texture metric; another option is to use the three metrics of horizontal gradient, vertical gradient and variance together and use "AND" logic for comprehensive judgment to enhance the ability to suppress weak texture regions.

[0079] In the embodiments of this specification, by introducing a comparison mechanism between texture complexity and effective threshold, the affine model is not ineffectively enabled in smooth regions lacking structural information, thereby reducing unnecessary computational overhead while ensuring reconstruction quality.

[0080] For example, in an indoor surveillance video, the current frame contains a white wall and a moving human body. In the preceding steps, the system identifies multiple initial affine motion regions, including the human arm region and a stationary block on the white wall. For the initial region on the white wall, the system calculates the pixel grayscale variance within its 8×8 area to be 12, the sum of horizontal gradients to be 8, and the sum of vertical gradients to be 6. Preset thresholds for effectiveness are: variance ≥ 50, horizontal gradient ≥ 30, and vertical gradient ≥ 30. Since none of these three indicators meet the thresholds, the system determines that the texture complexity of this region is too low and does not include it in the candidate affine motion regions. However, for the human arm region, its variance is 180, its horizontal gradient is 95, and its vertical gradient is 78, all exceeding the corresponding thresholds, and therefore it is identified as a candidate affine motion region. The entire selection process is based entirely on local statistical features, without relying on external information, achieving accurate preservation of effective affine regions and efficient removal of invalid regions.

[0081] Step 304: Obtain affine motion information of candidate affine motion regions and calculate affine processing cost based on affine motion information.

[0082] Affine motion information refers to the set of parameters describing how an image patch is mapped from a reference frame to the current frame under an affine transformation. It typically consists of six parameters and is used to express linear spatial transformations such as translation, rotation, scaling, and shearing. For example, a 2×3 affine transformation matrix [[a, b, c], [d, e, f]] is a typical representation of affine motion information, where (a, b, d, e) controls deformation and (c, f) controls translation.

[0083] Affine processing cost refers to the rate-distortion cost or prediction error metric incurred after motion compensation of candidate regions using an affine motion model. For example, after geometrically transforming a reference image block using affine motion information, the SATD value between the reference block and the current block is calculated, and combined with the number of bits required for affine parameter encoding, to form a comprehensive affine processing cost.

[0084] In practical applications, the system first extracts the corresponding positions of multiple control points from neighboring encoded blocks or reference frames for the selected candidate affine motion regions. Then, based on these corresponding points, it solves for the affine transformation parameters to obtain the affine motion information of the region. Specifically, the system can fit the positional relationships of at least three pairs of non-collinear points using the least squares method to calculate a complete six-parameter affine model. Next, the system uses this affine motion information to perform pixel-level geometric transformations on the corresponding regions in the reference frame, generating prediction blocks. Based on this, the system calculates the residual between the prediction blocks and the current candidate regions, and combines this with the encoding overhead of the affine parameters to obtain the affine processing cost. This cost can be compared with the aforementioned translation processing cost to determine the final motion model type to be adopted.

[0085] For example, in a video frame containing a person waving, the system identifies the arm region as a candidate affine motion region. Then, starting from the top-left, top-right, and bottom-left corners of this region, the system calculates the predicted positions of these three points in the reference frame using the motion vectors of their neighboring encoded blocks. Based on these three pairs of point coordinates, the system solves for a 2×3 affine transformation matrix using the least squares method, serving as the affine motion information for this region. Next, the system uses this matrix to resample the corresponding region in the reference frame, generating an affine prediction block. The SATD value between this prediction block and the current arm region is calculated to be 620, and the additional 12 bits required to encode the affine matrix are estimated, ultimately forming the affine processing cost. This cost is compared with the previously calculated translation processing cost (e.g., 850) to determine whether to switch to the affine model. The entire process achieves refined modeling and cost quantification of complex local motions.

[0086] Furthermore, obtaining affine motion information of the candidate affine motion region includes: dividing the candidate affine motion region into multiple sub-blocks according to size parameters; performing motion search on each sub-block to obtain motion information of each sub-block; and determining affine motion information based on the motion information of each sub-block.

[0087] The size parameters are preset configurations used to guide the sub-block division, including the width and height of the sub-blocks, such as 4×4, 8×8 or asymmetric sizes, to ensure that the number of sub-blocks after division is sufficient to support the parametric solution of the affine model.

[0088] Sub-blocks are local image units obtained by dividing candidate affine motion regions according to rules. Each sub-block participates in motion search independently, and its motion information reflects the displacement characteristics of that local position.

[0089] Motion information is the optimal displacement vector (MV) obtained by each sub-block through motion search in adjacent frames, which is usually represented by two-dimensional coordinates to indicate the offset in the horizontal and vertical directions.

[0090] Affine motion information consists of affine transformation parameters that describe the overall motion deformation of the entire candidate region. It is usually composed of a 2×3 matrix or six degrees of freedom parameters (such as a11, a12, a13, a21, a22, a23), and can be derived from the motion information of multiple sub-blocks through fitting methods such as least squares.

[0091] In practical applications, the system first divides the candidate affine motion region into several regularly arranged sub-blocks according to preset size parameters. For example, a 16×16 candidate region can be divided into four 8×8 sub-blocks or sixteen 4×4 sub-blocks. Then, the system performs an independent motion search for each sub-block within its search range of adjacent frames to obtain its optimal motion vector. This search can be based on cost functions such as SAD and SATD, and supports integer or sub-pixel precision. After obtaining the motion information of all sub-blocks, the system uses their spatial center coordinates and corresponding motion vectors as input, and solves for the global affine transformation parameters that best explain these local displacements using linear regression or least squares fitting methods, thus obtaining the affine motion information of the candidate region. The fitting process must satisfy the geometric constraints of the affine transformation, that is, the displacement of any point can be linearly expressed by a unified affine matrix.

[0092] For the step "perform motion search on each sub-block to obtain motion information of each sub-block", one possible approach is to have all sub-blocks share the same search range; another possible approach is to dynamically adjust the search window according to the position of the sub-block in the region to adapt to local motion differences.

[0093] For the step "determine affine motion information based on the motion information of each sub-block", one option is to use a six-parameter affine model for full-degree-of-freedom fitting; another option is to restrict it to a four-parameter simplified model (such as only containing rotation and scaling) to reduce computational complexity and improve numerical stability.

[0094] In the embodiments of this specification, parameterized modeling of complex non-rigid motion is achieved by aggregating and fitting sub-block-level motion information, providing a reliable motion representation for subsequent high-precision filtering or prediction.

[0095] For example, in a video clip showing a revolving door rotating, a 16×16 candidate affine motion region in the current frame covers the middle of the door. The system divides this region into four sub-blocks of 8×8 size. After performing a motion search on the upper left sub-block, the motion vector is obtained ( 3, 2); the top right sub-block is ( 5, 0); the bottom left sub-block is ( 1, 4); the bottom right sub-block is ( 3, 2). The system records the center coordinates of each sub-block (e.g., the center of the upper left sub-block is (4, 4)) and constructs a system of linear equations based on its motion vectors. By solving the least squares method, the system obtains a set of 2×3 affine parameters, which can accurately describe the clockwise rotational deformation of the entire region.

[0096] Furthermore, affine motion information is determined based on the motion information of each sub-block, including: determining the search range of each sub-block based on the direction of the translational motion information of the candidate affine motion region; performing motion search within the search range of each sub-block to obtain the refined motion information of each sub-block; and converting the refined motion information of each sub-block into affine motion information.

[0097] Translational motion information is the optimal motion vector (MV) obtained after performing a global translational motion search on the candidate region. It represents the overall displacement trend of the region and is used as the initial reference for sub-block search.

[0098] The search range is a local neighborhood window (e.g., ±4 pixels) set around the position pointed to by the translational motion information. This is used to limit the spatial range of motion search for each sub-block, avoid invalid searches, and improve convergence speed.

[0099] Refined motion information is a more accurate motion vector obtained by re-performing motion search within the limited search range of each sub-block, reflecting the details of local deformation.

[0100] The affine motion information is a 2×3 affine transformation matrix that ultimately describes the non-rigid motion of the entire candidate region, which is obtained by jointly fitting the refined motion information of multiple sub-blocks.

[0101] Example 1: Fitting method based on local search.

[0102] The system first divides the current filtering unit into multiple fixed-size sub-blocks (e.g., four 8×8 sub-blocks). Then, using the optimal motion vector obtained through block-wide translation search, the system determines the position of this vector in adjacent frames. Using this position as the center, a fine-grained motion search is performed for each sub-block within a preset small neighborhood (e.g., ±3 pixels). This process yields the refined motion vector for each sub-block. Next, the system treats the center coordinates of each sub-block and its corresponding refined motion vector as control point pairs, constructing a system of linear equations for the affine transformation. By solving this system of equations (e.g., using a pseudo-inverse method), the complete affine motion parameters can be obtained.

[0103] Example 2: Least squares-based fitting method.

[0104] This embodiment also divides the filtering unit into fixed-size sub-blocks and performs motion search independently on each sub-block (which can be limited to the translational MV neighborhood or a global search). After obtaining the motion vector of each sub-block, it is combined with the corresponding center coordinates to form an observation dataset. The system uses a least squares algorithm to minimize the sum of squared errors between the actual motion vectors of all sub-blocks and the predicted values ​​of the affine model. If some sub-blocks are located at texture edges or have low reliability, weights (such as those based on gradient magnitude) can be introduced to form a weighted least squares algorithm to further improve robustness.

[0105] Example 3: Fitting method based on feature point matching.

[0106] This embodiment does not rely on regular sub-block partitioning, but instead extracts corner points (such as Harris corner points or FAST feature points) with high response values ​​within the current filtering unit and its neighborhood. Subsequently, within the search region centered on the translational motion information location in the reference frame, feature descriptors (such as SIFT and ORB) are used for matching to obtain initial matching point pairs. To eliminate false matches, the system uses the Random Sample Consensus (RANSAC) algorithm to randomly select the minimum point set multiple times (3 pairs of points are sufficient to determine the affine transformation), calculates the transformation model, and counts the number of interior points. Finally, the model with the most interior points and the smallest reprojection error is selected as the affine motion information. This method is particularly suitable for scenes with occlusion, noise, or severe local deformation.

[0107] In the embodiments of this specification, the three implementation methods each have their advantages: Embodiment 1 is computationally efficient and suitable for real-time encoding; Embodiment 2 is mathematically rigorous and suitable for regions with uniform texture; Embodiment 3 is robust and suitable for complex dynamic scenes. The system can dynamically select the most suitable fitting strategy based on hardware resources, regional texture characteristics, or motion complexity.

[0108] For example, in a video containing rotating fan blades, the current 16×16 filter unit covers the middle of the blades. The system uses Embodiment 1: first, the entire translated MV is obtained as ( 2, 1), with this as the center, search within ±4 pixels for four 8×8 sub-blocks respectively, and obtain the refined MV as follows ( 4, 3), ( 1, 0), ( 5,2), (0, 1) Through linear fitting, the affine matrix obtained by the system can accurately describe the rotation and radial expansion of the blades, effectively improving the quality of subsequent filtering and reconstruction.

[0109] Step 306: If the translation processing cost and the affine processing cost satisfy the affine strategy, the affine motion information is determined as the target motion information of the candidate affine motion region.

[0110] Affine strategy refers to the decision rules used to determine whether to enable an affine motion model. It is usually based on the relative relationship between translation processing cost and affine processing cost or an absolute threshold setting. For example, when the affine processing cost is lower than the translation processing cost and the difference exceeds a preset threshold, the system determines that the affine strategy is satisfied.

[0111] Target motion information refers to the motion parameters ultimately used for motion compensation of the candidate region. It can be a translational motion vector or affine motion information. In this step, if the affine strategy is satisfied, the target motion information is the affine motion information obtained above.

[0112] In practical applications, after obtaining the translation processing cost and translation processing cost corresponding to the same candidate affine motion region, the system substitutes them into a preset affine strategy for judgment. Common affine strategies include: the affine processing cost is less than the translation processing cost; or the ratio of the affine processing cost to the translation processing cost is lower than a certain threshold (e.g., 0.9); or the reduction in the affine processing cost exceeds a fixed number of bits (e.g., saving more than 10 bits). If the judgment result satisfies the affine strategy, the system officially determines the affine motion information of that region as its target motion information for subsequent motion compensation and filtering operations; otherwise, the translation motion vector is retained as the target motion information. This decision-making process can be executed independently block by block, or it can be jointly optimized in conjunction with context information to improve overall consistency.

[0113] For example, in a video sequence containing a slowly advancing camera, the system identifies the edge region of a background building as a candidate affine motion region and calculates its translation processing cost as 780 and its affine processing cost as 650. The affine strategy adopted by the system is that "the affine processing cost must be lower than the translation processing cost and the difference must not be less than 50". Because 780... Since 650 = 130 > 50, the strategy is satisfied, so the system determines the previously solved 2×3 affine transformation matrix as the target motion information for this region. In the subsequent filtering stage, this region will generate a prediction block from the reference frame based on this affine parameter, instead of using a single translation vector. The entire decision-making process is based entirely on quantization cost comparison, ensuring the objectivity and reproducibility of model selection, while avoiding the misactivation of the affine model for static or simply moving regions.

[0114] While affine motion models can more accurately describe complex deformations, their computational cost is higher than that of translation models. Indiscriminately enabling affine processing may lead to resource waste. Therefore, the affine strategy further includes ensuring that the cost of scaling and translation is higher than the cost of adjusting the affine motion. When the translation and affine processing costs satisfy the affine strategy, the affine motion information is identified as the target motion information for candidate affine motion regions. This includes: obtaining error margin parameters and scaling factors, where the scaling factor is used for scaling and translation processing costs; obtaining the adjusted affine cost based on the affine processing cost and error margin parameters, and scaling the translation processing cost based on the scaling factor to obtain the scaled translation cost; and identifying the affine motion information as the target motion information for candidate affine motion regions when the scaled translation cost is higher than the adjusted affine cost.

[0115] The scaling factor is a coefficient used to scale the cost of translation processing. Its value is usually less than or equal to 1, and it is used to adjust the system's tolerance to the translation model. For example, the scaling factor is set to 0.9.

[0116] The error margin parameter is a non-negative value used to adjust the cost of affine processing upwards, compensating for additional errors introduced by interpolation, parameter estimation noise, etc., in the affine model. For example, the error margin parameter is set to 10.

[0117] The cost of scaling and translation is the product of the scaling factor and the translation processing cost, which is used to reduce the advantage weight of the translation model in cost comparison.

[0118] The adjusted affine cost is the sum of the affine processing cost and the error margin parameter, used to account for the inherent overhead of the affine model in decision-making.

[0119] In practical applications, the system first obtains the translation processing cost and affine processing cost corresponding to the candidate affine motion region. Then, the system reads the preset scaling factor and error margin parameter, and calculates the scaling translation cost and the adjusted affine cost, respectively. Specifically, the scaling translation cost equals the scaling factor multiplied by the translation processing cost, and the adjusted affine cost equals the affine processing cost plus the error margin parameter. The system compares the two values. If the scaling translation cost is greater than the adjusted affine cost, the current region is determined to meet the affine strategy conditions, and the affine motion information of that region is identified as its target motion information; otherwise, the translation motion information is retained as the target motion information.

[0120] For the step "obtaining error margin parameters and scaling factors", one option is to read fixed values ​​from the encoding configuration file; another option is to dynamically adjust the scaling factor and error margin parameters according to the current frame type, quantization parameters, or region size to adapt to the needs of different encoding scenarios.

[0121] In the embodiments of this specification, by introducing a scaling factor and an error margin parameter to normalize and tolerance-adjust the two types of costs, the affine decision not only depends on the original distortion value, but also takes into account the model complexity and system robustness, avoiding unnecessary model switching caused by small cost fluctuations.

[0122] Exemplarily, in a video frame containing a locally rotating object, the system identifies a 16×16 candidate affine motion region. The translation processing cost of this region is 450 (SATD based on the whole-block MV prediction), and the affine processing cost is 380 (SATD based on the fitting affine parameter prediction). The scaling factor configured by the system is 0.85, and the error margin parameter is 15. Accordingly, the scaled translation cost is calculated as 0.85×450 = 382.5, and the adjusted affine cost is 380 + 15 = 395. Since 382.5 < 395, the system determines that the affine strategy is not satisfied, so the affine motion information is not enabled. In another region, the translation processing cost is 500, and the affine processing cost is 360. With the same parameters, the scaled translation cost is 425, and the adjusted affine cost is 375. At this time, 425 > 375, the system determines that the affine strategy is satisfied, and determines the affine motion information of this region as the target motion information. The entire decision-making process is completely based on the cost comparison formula J_affine + δ < m·J_tran (where m ≤ 1), ensuring that the affine model is only activated when it brings sufficient reconstruction gain.

[0123] Step 308: Perform filtering processing based on the target motion information, the candidate affine motion region, and the reference image block corresponding to the candidate affine motion region to obtain an affine filtering sub-block, and obtain a filtered image based on each affine filtering sub-block and each translational filtering sub-block corresponding to the target image.

[0124] The reference image block refers to the corresponding image region in the reference frame that has spatio-temporal association with the current candidate region and is used to generate a prediction signal to support the filtering processing.

[0125] The affine filtering sub-block refers to the filtered output sub-block obtained by geometrically transforming the reference image block based on the affine motion information and then weighted-fusing it with the current region.

[0126] The translational filtering sub-block refers to the sub-block directly extracted from the reference image based on the translational motion vector and used for filtering, which is applicable to regions with simple motion behaviors.

[0127] In practical applications, the system first performs corresponding motion compensation operations based on the target motion information type of each candidate affine motion region. For regions where the target motion information is affine motion information, the system uses the affine parameters to perform pixel-level resampling of the reference image block to generate an affine prediction block. Then, the prediction block and the current region are weighted and averaged according to preset weights (such as inverse temporal distance ratio, adaptive residual weight, etc.) to obtain affine filter sub-blocks. For regions where the target motion information is still a translational motion vector, the corresponding reference image block is directly extracted from the reference frame as a translational prediction block, and weighted fusion is also performed to generate translational filter sub-blocks. All filter sub-blocks spatially cover the entire target image. The system stitches or overlaps these sub-blocks according to their positions to form a complete filtered image. During the fusion process, techniques such as weighted averaging of overlapping regions, boundary smoothing, or block artifact suppression can be used to ensure visual continuity.

[0128] For example, in a video frame containing a rotating windmill and a static background, the system uses an affine model to determine the windmill blade region, while retaining a translation model for the sky region. For the blade region, the system uses its 2×3 affine matrix to perform bilinear interpolation resampling on the corresponding reference image block from the previous frame, generating an affine prediction block, which is then fused with the current blade region with a weight of 0.6:0.4 to obtain an affine filtered sub-block. For the sky region, a reference image block is directly extracted from the same coordinate position in the previous frame and fused with a weight of 0.7:0.3 to obtain a translation filtered sub-block. After all sub-blocks cover the entire frame, the system performs Gaussian weighted smoothing on the overlapping boundaries of adjacent sub-blocks, finally synthesizing a complete filtered image. This image retains the clear edges of the rotating windmill structure while maintaining the low noise characteristics of the static region, demonstrating the advantages of multi-model collaborative filtering.

[0129] In video filtering tasks, directly applying uniform motion information to the entire candidate affine motion region makes it difficult to balance local motion consistency and texture detail fidelity. Therefore, a further approach is taken: the candidate affine motion region comprises multiple sub-blocks; filtering is performed based on target motion information, the candidate affine motion region, and the corresponding reference image block to obtain affine filter sub-blocks. This includes: determining the reference image block based on the target motion information; traversing each sub-block within the candidate affine motion region and calculating the ratio of the first motion search cost of each sub-block to the second motion search cost of the candidate affine motion region; adjusting the preset fusion weights of each sub-block inversely based on the ratio to obtain the target fusion weights for each sub-block; and using the fusion weights, weighted fusion of the pixel values ​​of each sub-block and their corresponding positions in the reference image block to generate affine filter sub-blocks.

[0130] The first motion search cost is the minimum distortion value obtained after performing a local motion search centered on the affine mapping position of a reference image block for a single sub-block, reflecting the reliability of the local matching of that sub-block.

[0131] The second motion search cost is the overall distortion value obtained by performing a global affine motion search on the entire candidate affine motion region, representing the overall motion fitting quality of the region.

[0132] The preset fusion weights are the default fusion coefficients assigned to each sub-block during system initialization. They are usually set to the same value (e.g., 0.5) to balance the initial contributions of the original pixels and the reference pixels.

[0133] The target fusion weight is a fusion coefficient that is dynamically adjusted based on the ratio of the search costs of the first and second motions, and is used to generate the final filtered result by weighting.

[0134] In practical applications, the system first generates a corresponding reference image block in the reference frame using affine interpolation based on the target motion information. Then, the system traverses each sub-block within the candidate affine motion region, performing a local motion search near its affine mapping position for each sub-block and calculating the first motion search cost for that sub-block. Simultaneously, the system obtains the second motion search cost recorded during the affine model fitting phase for that candidate region. Next, the system calculates the ratio of the first motion search cost to the second motion search cost and adjusts the preset fusion weight of the sub-block based on the inverse relationship of this ratio: the smaller the ratio, the better the local matching of the sub-block is compared to the overall fitting, and the higher its target fusion weight; conversely, the ratio is lowered. Finally, the system uses the target fusion weight of each sub-block to perform a weighted average of its original pixel values ​​and the pixel values ​​at the corresponding affine mapping positions in the reference image block, generating an affine filter sub-block for each sub-block, which is then stitched together to form a complete filtered output.

[0135] For the step "Calculate the first motion search cost of each sub-block", one option is to perform a full search within ±2 pixels around the affine mapping point; another option is to evaluate only at integer pixel positions without sub-pixel interpolation to reduce computational complexity.

[0136] In the embodiments of this specification, by introducing the ratio of local to global motion search costs to dynamically adjust the fusion weights, the filtering process can adaptively enhance the reference information of high-confidence sub-blocks and suppress interference from low-quality regions, thereby improving the ability to restore local details while maintaining the consistency of the affine structure.

[0137] For example, in a video frame containing a tilted moving vehicle, the system identifies a 16×16 candidate affine motion region and determines its target motion information as a set of six-parameter affine matrices. Based on this, the system generates a corresponding reference image block in the previous frame. This region is divided into four 8×8 sub-blocks. For the upper-left sub-block, the first motion search cost near its affine mapping position is 80; for the upper-right sub-block, it is 120; for the lower-left sub-block, it is 90; and for the lower-right sub-block, it is 200. The second motion search cost for this region is 100 (from the overall affine fitting stage). The system calculates the ratios of each sub-block as 0.8, 1.2, 0.9, and 2.0, respectively. The preset fusion weight is 0.5, and the adjustment factor k=1. After inverse proportional adjustment, the target fusion weights are approximately 0.56, 0.42, 0.53, and 0.25, respectively. Based on this, the system performs weighted fusion of the original pixels of each sub-block with their corresponding pixels in the reference image block: the upper left sub-block relies more on the reference pixels, while the lower right sub-block mainly retains the original pixels. Finally, the four affine filter sub-blocks are stitched together to form a complete filtering result, which preserves the overall tilting motion structure of the vehicle while avoiding blur artifacts caused by local occlusion.

[0138] Furthermore, the target fusion weight of each sub-block is obtained by adjusting the preset fusion weight of each sub-block inversely based on the ratio, including: adjusting the preset fusion weight of each sub-block inversely based on the ratio to obtain intermediate weights; calculating the confidence parameter of affine motion information, wherein the confidence parameter represents the degree of consistency between the affine motion model and the real motion; mapping the confidence to a preset numerical range to obtain weighting coefficients; and adjusting the intermediate weights based on the weighting coefficients to obtain the target fusion weight of each sub-block.

[0139] The preset fusion weight is the default fusion coefficient set for each sub-block during system initialization, usually taken as 0.5, which represents the initial equal weight fusion of the original pixel and the reference pixel.

[0140] The intermediate weight is the result of the initial adjustment of the preset fusion weight based on the inverse ratio relationship, reflecting the initial impact of local matching credibility on the fusion ratio.

[0141] The confidence parameter for affine motion information is an indicator that quantifies the consistency between the affine model and the actual motion. The higher the value, the more accurately the affine transformation describes the true deformation. For example, it can be calculated using the variance of the sub-block motion vector residuals, the control point fitting error, or the RANSAC inlier ratio.

[0142] The weighting coefficient is an adjustment factor obtained by normalizing the confidence parameter and mapping it to a preset numerical range (such as [0.7, 1.0]). It is used to scale the intermediate weights to reflect the macro-constraints on the overall model reliability.

[0143] The target fusion weight is the final fusion coefficient after secondary adjustment of the weighting coefficients, which is used to guide the pixel weighted fusion of each sub-block in the filtering process.

[0144] In practical applications, the system first calculates the ratio of the first motion search cost to the second motion search cost for each sub-block, and adjusts the preset fusion weights based on the inverse relationship of this ratio to obtain intermediate weights. Then, the system calculates the confidence parameter of the affine motion information. One possible approach is to use the mean squared error of the residuals between the refined motion vectors of each sub-block and the predicted vectors of the affine model; another possible approach is to use the condition number or least squares residual norm of the linear equation system during the fitting process as a confidence metric. Next, the system compresses this confidence parameter to a preset numerical range using a linear or nonlinear mapping function (such as piecewise linear mapping or the Sigmoid function) to obtain weighting coefficients. Finally, the system multiplies the intermediate weights by the weighting coefficients to obtain the target fusion weights for each sub-block, which are used in subsequent pixel-weighted fusion operations.

[0145] In the embodiments of this specification, by combining the local cost ratio and the global affine confidence to adjust the fusion weight, the filtering process can respond to changes in sub-block matching quality and avoid erroneous guidance when the overall model is inaccurate, thereby improving the robustness and adaptability of affine filtering.

[0146] For example, in a video frame containing a partially occluded rotating object, the system processes a 16×16 candidate affine motion region, dividing it into four 8×8 sub-blocks. The calculated first costs for each sub-block are 70, 90, 85, and 210, respectively, and the second cost is 100, resulting in ratios of 0.7, 0.9, 0.85, and 2.1. The preset fusion weight is 0.5, α=1, and the intermediate weights are approximately 0.59, 0.53, 0.54, and 0.24, respectively. Simultaneously, the system calculates the affine confidence parameter as 0.65 (out of 1.0) using the sub-block motion residuals, linearly mapping it to the interval [0.7, 1.0], resulting in a weighting coefficient of 0.78. The final target fusion weights for each sub-block are 0.59×0.78≈0.46, 0.53×0.78≈0.41, 0.54×0.78≈0.42, and 0.24×0.78≈0.19. Based on these weights, the system performs weighted fusion of the original pixels of each sub-block with their corresponding affine pixels in the reference image block. The lower right sub-block, due to poor local matching and low overall model confidence, primarily retains its original pixels, while other sub-blocks moderately fuse reference information to form high-quality affine filter sub-blocks.

[0147] In one embodiment of this specification, during the MCTF process, for regions with complex motion, affine motion information is introduced and compared with its translation processing cost. When the affine strategy is satisfied, the affine model is preferentially adopted, thereby more accurately characterizing non-translational motion (such as rotation, scaling, and perspective distortion). Based on this, the generated affine filter sub-blocks can better align with the reference image blocks, effectively improving motion compensation accuracy, reducing filter artifacts caused by motion model mismatch, and enhancing the robustness and reconstruction quality of image filtering in complex motion scenes.

[0148] The following is in conjunction with the appendix Figure 4 Taking the image filtering method provided in this specification as an example in the application of MCTF, the image filtering method will be further explained. Figure 4 This is a flowchart of an image filtering processing method provided in one embodiment of this specification, which specifically includes the following steps.

[0149] Step 402: Determine the search precision (1x, 2x, 4x downsampling layers, 1 / 2, 1 / 4, 1 / 8 pixel subdivision layers).

[0150] Step 404: Determine the filter unit division (e.g., uniformly divide according to a fixed 32x32 size under all precision).

[0151] Step 406: Determine the temporal reference frame queue (e.g., the N frames before and after the current frame to be filtered).

[0152] Step 408: Calculate the cost of translational motion.

[0153] Step 410: Determine if there is any potential affine motion.

[0154] Specifically, to determine whether potential affine motion exists, it is considered to exist if any one of the following five conditions is met: There are MVs whose final MV is an affine transformation at the same position in adjacent frames and within a nearby MxN range; The final MV of the adjacent cells in the left, top, and top-left directions at the current position is the MV of the affine transformation; The MV of the previous layer (L-1 layer) in the pyramid search at this position is the MV of the affine transformation; The MV translation cost of making a decision at this location in the previous layer (L-1 layer) of the pyramid search exceeds θ_L. 1; The current coding unit is not located in the first sub-block in the top left corner of the L-1 layer, and the difference between the coding unit MV corresponding to the top left sub-block position and the MV vector of the L-1 layer is greater than the threshold k.

[0155] Step 412: Determine if the texture complexity is below the effective threshold.

[0156] Specifically, if the current filtering unit is marked as a potential affine motion unit, the texture complexity of the current filtering unit is calculated. If the horizontal gradient, vertical gradient, or variance is lower than their respective effective thresholds, the affine transformation check of the current filtering unit is skipped, and step 426 is executed directly to perform filtering processing according to the translational motion model.

[0157] Step 414: If the current filter unit is marked as a potential affine motion unit, divide the current filter unit into a set of sub-blocks of fixed size.

[0158] Step 416: Fit the affine transformation MV.

[0159] Specifically, fitting an affine transformation MV can include any of the following three methods: The fitting method based on local search is as follows: the current filtering unit is divided into multiple sub-blocks of fixed size. Taking the position of the optimal motion vector obtained by the whole block translation search as the center, motion search is performed on each sub-block within a preset neighborhood range to obtain the refined motion vector of each sub-block. Then, the motion vectors of these sub-blocks are used as control points, and the overall affine transformation parameters are fitted by solving a system of linear equations, thereby obtaining the affine motion vector of the current filtering unit.

[0160] The least squares-based fitting method involves dividing the current filtering unit into multiple fixed-size sub-blocks, performing motion search on each sub-block to obtain its own motion vector, using the center coordinates of these sub-blocks and their corresponding motion vectors as observation data points, and employing the least squares algorithm or weighted least squares algorithm to solve for the affine transformation parameters that minimize the sum of squared fitting errors for all observation points, thereby obtaining the affine motion vector of the current filtering unit.

[0161] The fitting method based on feature point matching is as follows: corner points with significant texture features are extracted as feature points in the current filter block and its neighborhood. The matching position of each feature point is found in the corresponding search area of ​​the reference frame through the feature point matching algorithm. The motion offset between each pair of matching points is calculated. The matching point pairs are iteratively filtered using the random sampling consensus algorithm. After eliminating incorrect matches, robust affine transformation parameters are fitted to obtain the affine motion vector of the current filter unit.

[0162] Step 418: Calculate the affine cost and make a decision.

[0163] Specifically, calculate the reference cost after the affine transformation. If the cost J satisfies J_affine + δ <m If J_tran (m≤1) is used, then the MV is marked as an affine transformation MV, and all affine parameters are recorded. Subsequent filtering is performed using an affine processing model. Here, J_affine is the affine transformation reference cost, J_tran is the translation transformation reference cost, δ is the error margin, and m is a scaling factor less than or equal to 1. The role of m is to balance the computational cost of the affine transformation with the filtering performance. Otherwise, a translation processing model is used, and step 426 is directly executed for filtering.

[0164] Step 420: Calculate the reference filter block weights of the MCTF block using the cost of translational motion or the total cost of affine motion.

[0165] Step 422: Traverse all sub-blocks of affine motion, and inversely scale the reference weight of the reference filter block according to the ratio of the motion search cost of the sub-block to the motion search cost of the whole block.

[0166] Optionally, an additional affine motion confidence score can be calculated as a weighted average for the reference weight scaling (weighting the scaling magnitude). The calculation method is as follows:

[0167] Where η represents the confidence level of the affine transformation itself.

[0168] Step 424: Calculate the transformed pixel value of the current filter reference block based on the MV of the affine motion, and replace the filter reference buffer of the translational motion.

[0169] Step 426: Complete the processing of all filtering units, filtering precision, and reference frames, and output the filtered frame.

[0170] Through steps 402-426 above, a method is proposed that adaptively enables affine transformation during the MCTF pyramid search process and uses affine transformation information for weighted fusion in the MCTF motion compensation stage. This method employs a two-stage motion model selection process: first, translation estimation; then, affine estimation is triggered for candidate regions. The model is selected based on a filtering-related cost function, thereby improving alignment accuracy and noise reduction in complex motion scenarios.

[0171] Corresponding to the above method embodiments, this specification also provides embodiments of an image filtering processing apparatus. Figure 5 This is a schematic diagram of the structure of an image filtering processing device provided in one embodiment of this specification. Figure 5 As shown, the device includes: The filtering module 502 is configured to filter candidate affine motion regions in the target image and calculate the translation processing cost of the candidate affine motion regions.

[0172] The calculation module 504 is configured to acquire affine motion information of candidate affine motion regions and calculate the affine processing cost based on the affine motion information.

[0173] The determination module 506 is configured to determine the affine motion information as the target motion information of the candidate affine motion region when the translation processing cost and the affine processing cost satisfy the affine strategy.

[0174] The filtering module 508 is configured to perform filtering based on target motion information, candidate affine motion regions and corresponding reference image blocks to obtain affine filter sub-blocks, and obtain a filtered image based on each affine filter sub-block and each translation filter sub-block corresponding to the target image.

[0175] Optionally, the target image is the current frame; correspondingly, the filtering module 502 is further configured to divide the current frame into multiple filtering units for filtering processing; for the first filtering unit, first motion information of the corresponding position of the first filtering unit in the adjacent frame and the preset surrounding range of the corresponding position is obtained; second motion information of the first filtering unit in the current frame is obtained, wherein the adjacent frame is a reference frame that is adjacent to the current frame in the time sequence, and the first filtering unit is any filtering unit in the current frame; if there is affine motion information in the first motion information or the second motion information, the first filtering unit is determined as a candidate affine motion region.

[0176] Optionally, the target image is the current frame, and the current frame has an image pyramid containing multiple resolution levels; accordingly, the filtering module 502 is further configured to divide the current frame into multiple filtering units for filtering processing; for the first filtering unit of the target resolution level, it is detected whether there is a downsampling level at the target resolution level; if there is a downsampling level at the target resolution level and the downsampling level satisfies the affine activation strategy, then the region to which the first filtering unit belongs is marked as a candidate affine motion region.

[0177] Optionally, the target image is the current frame, and the current frame corresponds to a time-domain adjacent frame; accordingly, the filtering module 502 is further configured to perform motion search on the candidate affine motion region based on the translation motion model to obtain translation motion information; according to the translation motion information, a reference image block is extracted in the adjacent frame, and the pixel difference between the candidate affine motion region and the reference image block is calculated to obtain the translation processing cost.

[0178] Optionally, the calculation module 504 is further configured to divide the candidate affine motion region into multiple sub-blocks according to the size parameters; perform motion search on each sub-block to obtain the motion information of each sub-block; and determine the affine motion information based on the motion information of each sub-block.

[0179] Optionally, the calculation module 504 is further configured to determine the search range of each sub-block based on the direction of the translational motion information of the candidate affine motion region; perform motion search within the search range of each sub-block to obtain the refined motion information of each sub-block; and convert the refined motion information of each sub-block into affine motion information.

[0180] Optionally, the affine strategy includes a scaling translation cost that is higher than the adjusted affine cost; correspondingly, the determining module 506 is further configured to obtain an error margin parameter and a scaling factor, wherein the scaling factor is used for the scaling translation processing cost; based on the affine processing cost and the error margin parameter, the adjusted affine cost is obtained, and based on the scaling translation processing cost using the scaling factor, the scaling translation cost is obtained; if the scaling translation cost is higher than the adjusted affine cost, the affine motion information is determined as the target motion information of the candidate affine motion region.

[0181] Optionally, the filtering module 502 is further configured to filter initial affine motion regions in the target image; calculate the texture complexity of the initial affine motion regions; compare the texture complexity with the corresponding effective threshold, and determine the initial affine motion regions as candidate affine motion regions if the texture complexity is greater than or equal to the effective threshold.

[0182] Optionally, the candidate affine motion region includes multiple sub-blocks; correspondingly, the filtering module 508 is further configured to determine a reference image block based on the target motion information; traverse each sub-block within the candidate affine motion region and calculate the ratio of the first motion search cost of each sub-block to the second motion search cost of the candidate affine motion region; adjust the preset fusion weight of each sub-block based on the inverse ratio to obtain the target fusion weight of each sub-block; and use the fusion weight to perform weighted fusion of the pixel values ​​of each sub-block and the corresponding positions of each sub-block in the reference image block to generate affine filter sub-blocks.

[0183] Optionally, the filtering module 508 is further configured to adjust the preset fusion weights of each sub-block based on the inverse ratio to obtain intermediate weights; calculate the confidence parameter of the affine motion information, wherein the confidence parameter characterizes the consistency between the affine motion model and the real motion; map the confidence to a preset numerical range to obtain weighting coefficients; and adjust the intermediate weights based on the weighting coefficients to obtain the target fusion weights of each sub-block.

[0184] In this image filtering processing device, during the MCTF process, the device identifies candidate affine motion regions in the target image that may undergo complex motion through the screening module 502 and calculates their translation processing cost; the calculation module 504 further obtains the affine motion information of the region and calculates the corresponding affine processing cost; the determination module 506 compares the two costs according to a preset affine strategy, and determines the affine motion information as the target motion information when the conditions are met, thereby more accurately depicting non-translational motions such as rotation, scaling, or perspective; subsequently, the filtering module 508 filters the candidate regions and their reference image blocks based on the target motion information, generates more precisely aligned affine filter sub-blocks, and fuses all sub-blocks to output the final filtered image, effectively improving motion compensation accuracy, reducing artifacts caused by model mismatch, and enhancing the filtering robustness and reconstruction quality in complex motion scenes.

[0185] The above is a schematic scheme of an image filtering processing device according to this embodiment. It should be noted that the technical solution of this image filtering processing device and the technical solution of the image filtering processing method described above belong to the same concept. For details not described in detail in the technical solution of the image filtering processing device, please refer to the description of the technical solution of the image filtering processing method described above.

[0186] Figure 6 This is a structural block diagram of a computing device according to one embodiment of this specification. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0187] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0188] In one embodiment of this specification, the above-described components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0189] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0190] The processor 620 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described image filtering processing method.

[0191] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computing device embodiments are basically similar to the image filtering method embodiments, so the description is relatively simple; relevant parts can be referred to in the description of the image filtering method embodiments.

[0192] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described image filtering processing method.

[0193] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are basically similar to the image filtering method embodiments, so the description is relatively simple; relevant parts can be referred to in the description of the image filtering method embodiments.

[0194] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described image filtering processing method.

[0195] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the image filtering processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the image filtering processing method described above.

[0196] An embodiment of this specification also provides a method for storing a bitstream, comprising storing the bitstream in a storage medium, the bitstream being generated by the image filtering processing method described above.

[0197] The above is an illustrative scheme of a method for storing a bitstream according to this embodiment. It should be noted that the technical solution of this method belongs to the same concept as the technical solution of the image filtering processing method described above. For details not described in detail in the technical solution of the bitstream storage method, please refer to the description of the technical solution of the image filtering processing method described above.

[0198] An embodiment of this specification also provides a method for transmitting a bit stream, including transmitting a bit stream generated by the image filtering processing method described above.

[0199] The above is an illustrative scheme of a method for transmitting a bit stream according to this embodiment. It should be noted that the technical solution of this method belongs to the same concept as the technical solution of the image filtering processing method described above. For details not described in detail in the technical solution of the bit stream transmission method, please refer to the description of the technical solution of the image filtering processing method described above.

[0200] An embodiment of this specification also provides a computer-readable storage medium storing a bit stream generated by the image filtering processing method described above.

[0201] The above is an illustrative embodiment of a computer-readable storage medium. It should be noted that the technical solution of this computer-readable storage medium and the technical solution of the image filtering processing method described above belong to the same concept. Details not described in detail in the technical solution of the computer-readable storage medium can be found in the description of the technical solution of the image filtering processing method described above.

[0202] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0203] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0204] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0205] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0206] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image filtering processing method, characterized in that, include: Candidate affine motion regions are selected in the target image, and the translation processing cost of the candidate affine motion regions is calculated. Obtain the affine motion information of the candidate affine motion region, and calculate the affine processing cost based on the affine motion information; If the translation processing cost and the affine processing cost satisfy the affine strategy, the affine motion information is determined as the target motion information of the candidate affine motion region; Based on the target motion information, the candidate affine motion region and the reference image block corresponding to the candidate affine motion region, filtering is performed to obtain affine filter sub-blocks, and based on each affine filter sub-block and each translation filter sub-block corresponding to the target image, a filtered image is obtained.

2. The method according to claim 1, characterized in that, The target image is the current frame; the step of filtering candidate affine motion regions in the target image includes: The current frame is divided into multiple filtering units for filtering processing; For the first filtering unit, first motion information of the first filtering unit at the corresponding position in the adjacent frame and within a preset surrounding range of the corresponding position is obtained; Obtain the second motion information of the first filtering unit in the current frame that is adjacent to the filtering unit, wherein the adjacent frame is a reference frame that is adjacent to the current frame in the time sequence, and the first filtering unit is any filtering unit in the current frame; If affine motion information exists in the first motion information or the second motion information, the first filtering unit is determined as a candidate affine motion region.

3. The method according to claim 1, characterized in that, The target image is the current frame, and the current frame has an image pyramid containing multiple resolution levels; the process of filtering candidate affine motion regions in the target image includes: The current frame is divided into multiple filtering units for filtering processing; For the first filtering unit of the target resolution level, it is detected whether the target resolution level has a downsampling level; If a downsampling level exists at the target resolution level and the downsampling level satisfies the affine activation strategy, then the region to which the first filtering unit belongs is marked as a candidate affine motion region.

4. The method according to any one of claims 1-3, characterized in that, The target image is the current frame, and the current frame corresponds to a time-domain adjacent frame; The calculation of the translation processing cost of the candidate affine motion region includes: Based on the translational motion model, a motion search is performed on the candidate affine motion region to obtain translational motion information; Based on the translation motion information, a reference image block is extracted from the adjacent frames, and the pixel difference between the candidate affine motion region and the reference image block is calculated to obtain the translation processing cost.

5. The method according to claim 1, characterized in that, The step of obtaining the affine motion information of the candidate affine motion region includes: The candidate affine motion region is divided into multiple sub-blocks according to the size parameters; Perform motion search on each sub-block to obtain motion information for each sub-block; Affine motion information is determined based on the motion information of each sub-block.

6. The method according to claim 5, characterized in that, The determination of affine motion information based on the motion information of each sub-block includes: Based on the direction of the translational motion information of the candidate affine motion region, the search range of each sub-block is determined; Motion search is performed within the search range of each sub-block to obtain refined motion information of each sub-block; The refined motion information of each sub-block is converted into affine motion information.

7. The method according to claim 1, characterized in that, The affine strategy includes a higher translation cost after scaling than an adjusted affine cost; the step of determining the affine motion information as the target motion information of the candidate affine motion region when the translation processing cost and the affine processing cost satisfy the affine strategy includes: Obtain the error margin parameter and the scaling factor, wherein the scaling factor is used to scale the translation processing cost; Based on the affine processing cost and the error margin parameter, the adjusted affine cost is obtained, and the translation processing cost is scaled based on the scaling factor to obtain the scaled translation cost. If the translation cost after scaling is higher than the affine cost after adjustment, the affine motion information is determined as the target motion information of the candidate affine motion region.

8. The method according to claim 1, characterized in that, The step of filtering candidate affine motion regions in the target image includes: Select the initial affine motion region in the target image; Calculate the texture complexity of the initial affine motion region; The texture complexity is compared with the corresponding effective threshold. If the texture complexity is greater than or equal to the effective threshold, the initial affine motion region is determined as the candidate affine motion region.

9. The method according to claim 1, characterized in that, The candidate affine motion region includes multiple sub-blocks; the filtering process based on the target motion information, the candidate affine motion region, and the corresponding reference image block to obtain affine filter sub-blocks includes: A reference image block is determined based on the target motion information; Traverse each sub-block within the candidate affine motion region and calculate the ratio of the first motion search cost of each sub-block to the second motion search cost of the candidate affine motion region; Based on the ratio, the preset fusion weights of each sub-block are adjusted inversely to obtain the target fusion weights of each sub-block; Using the fusion weights, the pixel values ​​of each sub-block and the corresponding positions of each sub-block in the reference image block are weighted and fused to generate affine filter sub-blocks.

10. The method according to claim 9, characterized in that, The step of adjusting the preset fusion weights of each sub-block inversely based on the ratio to obtain the target fusion weights of each sub-block includes: The preset fusion weights of each sub-block are adjusted inversely based on the ratio to obtain the intermediate weights; Calculate the confidence parameter of the affine motion information, wherein the confidence parameter characterizes the degree of consistency between the affine motion model and the real motion; The confidence level is mapped to a preset numerical range to obtain a weighting coefficient; The intermediate weights are adjusted based on the weighting coefficients to obtain the target fusion weights for each sub-block.

11. An image filtering processing device, characterized in that, include: The filtering module is configured to filter candidate affine motion regions in the target image and calculate the translation processing cost of the candidate affine motion regions. The calculation module is configured to acquire affine motion information of the candidate affine motion region and calculate the affine processing cost based on the affine motion information. The determination module is configured to determine the affine motion information as the target motion information of the candidate affine motion region when the translation processing cost and the affine processing cost satisfy the affine strategy. The filtering module is configured to perform filtering based on the target motion information, the candidate affine motion region and the reference image block corresponding to the candidate affine motion region to obtain affine filter sub-blocks, and obtain a filtered image based on each affine filter sub-block and each translation filter sub-block corresponding to the target image.

12. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the image filtering processing method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, It stores a computer program / instruction that, when executed by a processor, implements the steps of the image filtering processing method according to any one of claims 1-10.

14. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the image filtering processing method according to any one of claims 1-10.

15. A method for storing a bit stream, comprising storing the bit stream in a storage medium, characterized in that, The bitstream is generated by the image filtering processing method according to any one of claims 1-10.

16. A method for transmitting a bit stream, comprising transmitting the bit stream, characterized in that, The bitstream is generated by the image filtering processing method according to any one of claims 1-10.

17. A computer-readable storage medium storing a bit stream thereon, characterized in that, The bitstream is generated by the image filtering processing method according to any one of claims 1-10.

Images