Time domain filtering method and device, equipment, storage medium and program product

Abstract

The invention relates to a time domain filtering method and device, equipment, a storage medium and a program product. The method comprises the following steps: determining a filtering block corresponding to an original video frame to be filtered; the texture complexity of the filtering block is determined, an intensity representation value corresponding to the screen content intensity of the filtering block is determined, the screen content intensity is used for reflecting the complexity of the content in the filtering block, and the texture complexity is used for reflecting the complexity of the texture in the filtering block; according to the texture complexity and / or the intensity characterization value, a search deviation value of the filtering block is determined, and the search deviation value is used for reflecting the deviation size of motion information search according to the size of the filtering block; expanding the size of the filtering block according to the search deviation value to obtain the expanded size of the filtering block; and performing motion information search on the filtering block according to the extension size, wherein a motion information search result is used for performing time domain filtering on the original video frame. By adopting the method, the search efficiency and the filtering effect can be improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a temporal filtering method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the development of internet and computer technology, video coding technology has continuously evolved and progressed. In video coding, motion-compensated temporal filtering (MCTF) can be used to perform temporal filtering on video before encoding. MCTF is a standardized motion-compensated temporal filtering method that enhances the compression performance of video sequences and improves video quality by estimating and compensating for motion between frames.

[0003] In traditional techniques, motion-compensated temporal filtering typically involves dividing the video into fixed-size filter blocks for motion information search and filtering.

[0004] However, using fixed-size filter blocks for motion information search and filtering can easily affect the accuracy of motion information search, thus affecting the filtering effect. Summary of the Invention

[0005] Therefore, it is necessary to provide a time-domain filtering method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the accuracy of motion information search and thus enhance the filtering effect, in order to address the above-mentioned technical problems.

[0006] On one hand, this application provides a temporal filtering method, comprising: determining a filter block corresponding to an original video frame to be filtered; determining the texture complexity of the filter block and determining an intensity characterization value corresponding to the screen content intensity of the filter block, wherein the screen content intensity is used to reflect the complexity of the content in the filter block and the texture complexity is used to reflect the complexity of the texture in the filter block; determining a search deviation value of the filter block based on the texture complexity and / or the intensity characterization value, wherein the search deviation value is used to reflect the magnitude of the deviation in motion information search according to the size of the filter block; expanding the size of the filter block according to the search deviation value to obtain an expanded size of the filter block; performing motion information search on the filter block according to the expanded size, wherein the result of the motion information search is used to perform temporal filtering on the original video frame.

[0007] On the other hand, this application also provides a temporal filtering device, comprising: a block determination module, configured to determine a filter block corresponding to an original video frame to be filtered; an information determination module, configured to determine the texture complexity of the filter block and determine an intensity characterization value corresponding to the screen content intensity of the filter block, wherein the screen content intensity reflects the complexity of the content in the filter block and the texture complexity reflects the complexity of the texture in the filter block; a deviation determination module, configured to determine a search deviation value of the filter block based on the texture complexity and / or the intensity characterization value, wherein the search deviation value reflects the magnitude of the deviation in motion information search according to the size of the filter block; a size expansion module, configured to expand the size of the filter block according to the search deviation value to obtain an expanded size of the filter block; and a search module, configured to perform motion information search on the filter block according to the expanded size, wherein the result of the motion information search is used to perform temporal filtering on the original video frame.

[0008] On the other hand, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described time-domain filtering method.

[0009] On the other hand, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described time-domain filtering method.

[0010] On the other hand, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described time-domain filtering method.

[0011] The aforementioned temporal filtering method, apparatus, computer equipment, computer-readable storage medium, and computer program product determine the filter block corresponding to the original video frame to be filtered, determine the texture complexity of the filter block, and determine the intensity characterization value corresponding to the screen content intensity of the filter block. The screen content intensity reflects the complexity of the content in the filter block, and the texture complexity reflects the complexity of the texture in the filter block. Based on the texture complexity and / or intensity characterization value, a search deviation value for the filter block is determined. The search deviation value reflects the magnitude of the deviation in motion information search based on the size of the filter block. The size of the filter block is expanded based on the search deviation value to obtain the expanded size of the filter block. Motion information search is performed on the filter block according to the expanded size, and the result of the motion information search is used for temporal filtering of the original video frame. Since the deviation or error of motion information search based on the size of the filter block is relatively large when the filter block is complex, performing motion information search on the filter block according to the expanded size (larger than the filter block size) can reduce the deviation and improve the accuracy of motion information search, thereby improving the overall filtering effect. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a diagram illustrating the application environment of a time-domain filtering method in one embodiment;

[0014] Figure 2 This is a schematic diagram of a motion-assisted temporal filtering method in one embodiment.

[0015] Figure 3 This is a schematic diagram of a pyramid structure in one embodiment;

[0016] Figure 4 This is a flowchart illustrating a time-domain filtering method in one embodiment;

[0017] Figure 5 These are video frames at different scales in one embodiment;

[0018] Figure 6 This is a schematic diagram of a Gaussian kernel in one embodiment;

[0019] Figure 7 This is a block diagram of a time-domain filtering device in one embodiment;

[0020] Figure 8 This is an internal structural diagram of a computer device in one embodiment;

[0021] Figure 9 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] The time-domain filtering method provided in this application embodiment can be applied to, for example... Figure 1The application environment shown includes a computer device 102 and a user terminal 104. The computer device 102 communicates with the user terminal 104 via a network. A data storage system can store the data that the computer device 102 needs to process. The data storage system can be integrated onto the computer device 102 or located in the cloud or on another network server.

[0024] Specifically, computer device 102 can receive videos uploaded by other devices, perform temporal filtering on the videos, encode the temporally filtered videos, and send the encoded videos to user terminal 104. Alternatively, user terminal 104 can actively request encoded videos from computer device 102. Before encoding the videos, computer device 102 can decode the videos, i.e., it can transcode the videos (decode and then encode), and send the transcoded videos to user terminal 104. The videos can be, but are not limited to, short videos, live videos, movies, or TV series. Computer device 102 performs temporal filtering on each video frame in the video to obtain a temporally filtered video, and then encodes the temporally filtered video. In this application, to improve filtering efficiency, computer device 102 determines the filter block corresponding to the original video frame to be filtered. Computer device 102 determines the texture complexity of the filter block and the intensity characterization value corresponding to the screen content intensity of the filter block. The screen content intensity reflects the complexity of the content in the filter block, and the texture complexity reflects the complexity of the texture in the filter block. Based on the texture complexity and / or intensity characterization value, computer device 102 determines the search deviation value of the filter block. The search deviation value reflects the magnitude of the deviation in motion information search based on the size of the filter block. Computer device 102 expands the size of the filter block based on the search deviation value to obtain an expanded size of the filter block. Motion information search is then performed on the filter block according to the expanded size. The result of the motion information search is used for temporal filtering of the original video frame. Since the accuracy of motion information search based on the filter block is low and the error is large when the filter block is complex, using an expanded filter block larger than the original filter block for motion information search can improve the accuracy and efficiency of motion information search, thereby contributing to improved overall filtering efficiency.

[0025] The computer device 102 can be a terminal or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The cloud server provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal can be, but is not limited to, desktop computers, laptops, smartphones, and tablets. The computer device 102 and the user terminal 104 can be connected directly or indirectly via wired or wireless communication, which is not limited herein.

[0026] The temporal filtering involved in this application can be, but is not limited to, motion-compensated temporal filtering, such as MCTF (Motion Compensated Temporal Filter). MCTF is a temporal motion compensation filtering technique that improves video compression performance and quality by estimating and compensating for motion between frames. Motion-compensated temporal filtering is a technique used in video coding systems to enhance signal processing in the temporal domain of a video sequence through motion information between frames. Its main purpose is to reduce temporal redundancy in the video, improve video compression efficiency, and maintain or even improve video quality. Figure 2 The diagram illustrates a principle of motion-compensated temporal filtering, which occurs between lookahead and encoding, comprising motion estimation, motion compensation, and temporal filtering. Motion estimation: A fixed-size block matching algorithm and search modes such as Full Search, Three-Step Search, and Pyramid Search are used to determine the motion trajectory of each pixel or block between adjacent frames, recording it as a motion vector (MV) in the MCTF (Multi-Channel Graphical Function). In video coding, motion vectors represent the movement of an image block, such as a coded block, from one frame to another. Vector values ​​typically include horizontal and vertical information, representing the displacement of the block on the time axis within a frame. To balance search accuracy and computational complexity, the encoder can employ a pyramid structure built with different downsampling rates, expanding the search from high downsampling rates to high precision. Motion estimation includes multi-layer integer-pixel motion estimation and multi-layer sub-pixel search (sub-pixel motion estimation), for example… Figure 2It includes three layers of integer-pixel motion estimation and three layers of sub-pixel motion estimation. The integer-pixel motion estimation process employs a pyramid-shaped hierarchical downsampling search. For example, the pyramid has three layers, each corresponding to a resolution; the bottom layer corresponds to frames at the original resolution, and the top layer corresponds to frames at the highest downsampling rate. The pyramid structure is as follows: Figure 3 As shown, layer L0 corresponds to the original video frame, layer L1 corresponds to the video frame after 2x downsampling, and layer L2 corresponds to the video frame after 4x downsampling. After motion estimation, the computer device can perform motion-compensated temporal filtering on the original video frame based on the motion estimation results to obtain the filtered video frame. Motion compensation: Adjusting the pixel values ​​of the current frame according to motion information, using motion vectors to locate and compensate the pixel positions within the frame, aligning homogeneous blocks on the current frame and the reference frame. Temporal filtering: Calculating the weights of homogeneous blocks on different frames using a series of similarity indices, and using bidirectional or unidirectional filtering to filter the motion-compensated frame, removing short-term fluctuations and noise, thereby improving the visual quality of the video. The filtering strength of the temporal filtering block can be determined based on the reference block, block prediction cost, reference frame distance, and reference frame weight, and then weighted fusion is performed based on the filtering strength (the calculation formula for temporal filtering) to obtain the filtered video frame. MCTF can effectively remove redundant information in the temporal domain, improve the reference efficiency of temporal information during encoding, improve the efficiency of video compression, and achieve higher quality video transmission within a given bandwidth. By compensating for and filtering noise introduced by motion, MCTF technology ensures smooth video transitions and reduces blockiness and glitches. It can adapt to video content at various resolutions and frame rates, making it suitable for applications such as live streaming.

[0027] In some embodiments, such as Figure 4 As shown, a time-domain filtering method is provided. This method can be executed by a terminal or a server, or by both a terminal and a server. This method can be applied to… Figure 1 Taking computer device 102 as an example, the following steps are included:

[0028] Step 402: Determine the filter block corresponding to the original video frame to be filtered.

[0029] The original video frame can be any video frame in the video to be filtered. For example, the original video frame can be a video frame in a GOP (Group of Pictures). A GOP consists of a series of consecutive video frames, and a COP can include keyframes (i.e., intra-predictive frames, I-frames), forward-predictive frames, and / or bidirectional predictive frames (B-frames). For example, the first frame in a COP is a keyframe (intra-predictive frame, I-frame) or a forward-predictive frame, and subsequent frames are bidirectional predictive frames (B-frames).

[0030] Before filtering the original video frames, multi-level integer-pixel motion estimation and multi-level subpixel search can be performed (e.g., integer-pixel motion estimation and subpixel search in MCTF). In multi-level integer-pixel motion estimation of the original video frames, each level uses original video frames of different scales. Figure 2 Taking the 3-layer integer pixel motion estimation as an example, integer pixel motion estimation is first performed using video frames downsampled by 4x, then using video frames downsampled by 2x, and finally using the original video frames. The search block size used in each layer of integer pixel motion estimation is the same. In this application, the filter block can be the search block used in each layer of integer pixel motion estimation. Figure 5 As shown, the original video frame is downsampled by a factor of 2 to obtain downsampled video frame 1, and downsampled video frame 1 is downsampled by a factor of 2 to obtain downsampled video frame 2. Then, downsampled video frame 2 is used for integer-pixel motion estimation, followed by downsampled video frame 1, and finally, the original video frame is used for integer-pixel motion estimation. Taking an 8×8 filter block size as an example... Figure 5 In the original video frame, each 8×8 image block is a filter block, for example, image block A is a filter block. In downsampled video frame 1, each 8×8 image block is a filter block, for example, image block B is a filter block. In downsampled video frame 2, each 8×8 image block is a filter block, for example, image block C is a filter block.

[0031] During the pixel-by-pixel search process, search blocks can be obtained by inserting pixels into the original video frames. The size of the search blocks used in the pixel-by-pixel search process is the same as the size of the search blocks used in integer-pixel motion estimation. Filter blocks can also be used in the pixel-by-pixel search process; a search block is an image block whose motion vector needs to be determined.

[0032] Step 404: Determine the texture complexity of the filter block and determine the intensity characterization value corresponding to the screen content intensity of the filter block. The screen content intensity is used to reflect the complexity of the content in the filter block, and the texture complexity is used to reflect the complexity of the texture in the filter block.

[0033] Texture complexity and screen content intensity are used to reflect the complexity of the filter block. Texture complexity reflects the complexity of the texture of the filter block. The original video frame has three channels: YUV, where Y represents luminance, and U and V are the two chrominance channels.

[0034] For example, a computer device can determine the texture complexity of a filter block based on the pixel values ​​in the filter block, and can calculate the texture complexity separately for each channel. The texture complexity calculated for the luminance channel can be called the luminance texture complexity, and the texture complexity calculated for the chrominance channel can be called the chrominance texture complexity. The luminance texture complexity can be expressed, for example, as... Chromaticity texture complexity, for example, includes the first chroma texture complexity calculated under the U channel. The complexity of the second chroma texture calculated under the V channel.

[0035] For example, a computer device can calculate the texture complexity of the original video frame and use that texture complexity as the texture complexity of the filter block. Alternatively, the original video frame can be divided into multiple image regions, the texture complexity of each image region can be calculated, the image region corresponding to the filter block can be determined, and the texture complexity of the image region corresponding to the filter block can be used as the texture complexity of that filter block. Here, the image region corresponding to the filter block contains the filter block or a downsampled image block containing the filter block.

[0036] For example, a computer device can calculate the texture complexity of the video frame to which the filter block belongs, and use the texture complexity of the video frame to which the filter block belongs as the texture complexity of the filter block. For instance, if the video frame to which the filter block belongs is a downsampled video frame obtained by downsampling the original video frame, then the texture complexity of the downsampled video frame is used as the texture complexity of the filter block.

[0037] For example, any downsampled video frame can be divided into multiple image regions. For instance, the smallest sampled video frame can be divided into multiple image regions. The texture complexity of each image region is calculated, the image region corresponding to the filter block is determined, and the texture complexity of the image region corresponding to the filter block is used as the texture complexity of the filter block. The image region corresponding to the filter block contains the filter block or a downsampled image block containing the filter block.

[0038] For example, screen content intensity is used to reflect the complexity of the content in a filter block. For instance, the more details in a filter block, the higher the screen content intensity; the more complex the scene in a filter block, the higher the screen content intensity. Screen content intensity can be divided into three categories: high, medium, and weak. The intensity characterization value is used to quantify the screen content intensity. The intensity characterization value can be calculated under the luminance channel, or it can be calculated separately for each channel. For example, if the intensity characterization value is less than or equal to a first intensity threshold, the screen content intensity is weak; if the intensity characterization value is greater than the first intensity threshold and less than a second intensity threshold, the screen content intensity is medium; if the intensity characterization value is greater than or equal to the second intensity threshold, the screen content intensity is high. The first intensity threshold can be represented by δ1, and the second intensity threshold can be represented by δ2. The first and second intensity thresholds can be set according to actual needs, for example, δ1 = 1, δ2 = 4. The screen content intensity can be expressed as:

[0039]

[0040] Where R represents the filter block, C scc SCC(R) represents the intensity of the screen content. StrongScc represents high screen content intensity, WeakScc represents medium screen content intensity, and NonScc represents weak screen content intensity.

[0041] For example, in integer pixel motion estimation or subpixel search, motion estimation or search can be performed separately in the three YUV channels. The screen content intensity can be calculated in the Y channel, or the screen content intensity can be calculated separately for each channel.

[0042] Step 406: Determine the search deviation value of the filter block based on the texture complexity and / or intensity characterization value. The search deviation value is used to reflect the magnitude of the deviation in searching for motion information according to the size of the filter block.

[0043] The search deviation value is positively correlated with the intensity characterization value. "Performing motion information search" refers to the process of obtaining motion vectors through motion estimation. The search deviation value can be understood as the error probability, which reflects the magnitude of the error when performing motion information search using the size of the filter block. Motion information search refers to performing motion estimation or pixel-by-pixel search to determine the motion vector of the filter block.

[0044] For example, the search bias value of the filter block can be determined based on the luminance texture complexity and / or chrominance texture complexity, and the search bias value is negatively correlated with the luminance texture complexity and positively correlated with the chrominance texture complexity.

[0045] For example, the search bias value of the filter block can be determined based on the luma texture complexity and / or chroma texture complexity and intensity characterization value. For instance, the search bias value of the filter block can be determined based on the luma texture complexity and intensity characterization value, and the search bias value is negatively correlated with the luma texture complexity. Alternatively, the search bias value of the filter block can be determined based on the chroma texture complexity and intensity characterization value, and the search bias value is positively correlated with the chroma texture complexity.

[0046] Step 408: Expand the size of the filter block according to the search deviation value to obtain the expanded size of the filter block.

[0047] For example, the expansion size is positively correlated with the search deviation value. For instance, the expansion ratio of the filter block can be determined based on the search deviation value. The expansion ratio is positively correlated with the search deviation value. The size of the filter block can be expanded according to the expansion ratio to obtain the expansion size of the filter block.

[0048] For example, the length and width of the filter block can be the same, and the size of the filter block can be set as needed, for example, an 8×8 image block. Of course, the length and width of the filter block can also be different.

[0049] For example, the length of the filter block can be expanded according to the expansion ratio to obtain the expanded length, and the width of the filter block can be expanded according to the expansion ratio to obtain the expanded width. The expanded length and the expanded width are used as the expansion size of the filter block.

[0050] Step 410: Perform motion information search on the filter block according to the extended size. The results of the motion information search are used to perform temporal filtering on the original video frame.

[0051] For example, an extended filter block corresponding to a filter block can be determined based on the extended size, and motion information can be searched using the extended filter block. The extended filter block contains the filter block.

[0052] For example, the extended filter block can be determined from the video frame to which the filter block belongs according to the extended size. For instance, if the filter block is an image block in the original video frame, the extended filter block can be obtained by determining the region in the original video frame that has the extended size and contains the filter block.

[0053] In the aforementioned temporal filtering method, the filter block corresponding to the original video frame to be filtered is determined, the texture complexity of the filter block is determined, and the intensity representation value corresponding to the screen content intensity of the filter block is determined. The screen content intensity reflects the complexity of the content in the filter block, and the texture complexity reflects the complexity of the texture in the filter block. Based on the texture complexity and / or intensity representation value, the search deviation value of the filter block is determined. The search deviation value reflects the magnitude of the deviation in motion information search based on the size of the filter block. The size of the filter block is expanded based on the search deviation value to obtain the expanded size of the filter block. Motion information search is performed on the filter block according to the expanded size, and the result of the motion information search is used for temporal filtering of the original video frame. Since the deviation or error of motion information search based on the size of the filter block is relatively large when the filter block is complex, performing motion information search on the filter block according to the expanded size, which is larger than the size of the filter block, can reduce the deviation and improve the accuracy of motion information search, thereby improving the overall filtering effect.

[0054] Currently, the implementation of MCTF suffers from increased computational load and easy accumulation of search errors due to the multi-level complexity of the pyramid search process and the uncertainty of inter-frame motion. This impacts video coding efficiency and quality. The temporal filtering method provided in this application can be applied to MCTF to improve the efficiency and accuracy of motion information search, thereby improving overall coding efficiency and quality.

[0055] MCTF typically uses fixed-size temporal filtering units (i.e., filter blocks) in a pyramid-shaped search structure to acquire motion information. In fast-moving or complex motion scenarios, search efficiency is easily affected by the search range and accuracy. The temporal filtering method provided in this application, applied to MCTF, improves the search accuracy and efficiency of motion vectors by adaptively expanding the size of the filter blocks (also called filter units). This reduces the risk of getting trapped in local optima. Adaptively expanding the filter block size also improves the accuracy of MCTF parameter calculations, increases search efficiency, reduces the probability of error accumulation, enhances overall coding efficiency, reduces computational costs, and improves video quality.

[0056] The temporal filtering method provided in this application can be applied to any scenario requiring video encoding, including but not limited to short video and live streaming scenarios. In short video scenarios, videos uploaded by creators need to be transcoded by the server before being sent to users for viewing. Video encoding is required during the transcoding process. During video encoding, it is necessary to complete the encoding as quickly as possible while maintaining compression efficiency. For example, in live streaming scenarios, to reduce stuttering and make the live stream smoother, the encoder needs to encode the captured video at the fastest possible speed, thus requiring an increase in the encoding speed of the video encoder. The temporal filtering method provided in this application enables the video encoding process to complete the encoding faster while maintaining compression efficiency, allowing users to watch the creator's video as soon as possible. Furthermore, the temporal filtering method provided in this application allows the video encoder to perform temporal filtering at a faster speed, thereby improving overall encoding efficiency, reducing stuttering, and making the live stream smoother.

[0057] The temporal filtering method provided in this application can be related to VVC (Versatile Video Coding). VVC is the latest generation video coding standard developed by the International Telecommunication Union. It is designed to provide more efficient video compression technology for different types of network environments and various resolutions, improving encoding and decoding efficiency by approximately 50% compared to its predecessor, H.265 (HEVC).

[0058] In some embodiments, the motion estimation process of the original video frame includes multi-layer integer-pixel motion estimation and multi-layer sub-pixel search. The filter block is the search block used in the integer-pixel motion estimation or the search block used in the sub-pixel search. Each layer of integer-pixel motion estimation uses video frames of different scales from the original video frame. The texture complexity of the filter block is determined, including any of the following: determining the texture complexity of the filter block based on the pixel values ​​in the filter block; determining image regions that have a mapping relationship with the filter block from video frames of different scales, using the texture complexity of the image regions as the texture complexity of the filter block, and the image regions containing the filter block or the image regions containing downsampled image blocks of the filter block.

[0059] The video frames at different scales include the original video frames and downsampled video frames of the original video frames. The downsampled image block of the filter block refers to the result obtained by downsampling the filter block.

[0060] For example, the image region and the filter block belong to the same video frame, and the image region contains the filter block.

[0061] For example, the scale of the video frame to which the image region belongs is smaller than that of the video frame to which the filter block belongs. For instance, the filter block belongs to the original video frame, the image region belongs to the downsampled video frame, and the image region contains the downsampled image block of the filter block in that downsampled video frame. Figure 5 In the first downsampling step, the 32×32 image region in the original video frame becomes a 16×16 image region in downsampled video frame 1. After another 2x downsampling step, the 32×32 image region in the original video frame becomes an 8×8 image region in downsampled video frame 2. Therefore, if the filter block is image block A, the texture complexity of image block A can be determined based on the texture complexity of image block B or the image region containing image block B. Similarly, the texture complexity of image A can be determined using the texture complexity of image block C or the image region containing image block C. Likewise, if the filter block is image block B, the texture complexity of image A can be determined using the texture complexity of image block C or the image region containing image block C.

[0062] In this embodiment, the texture complexity can be calculated directly based on the pixel values ​​of the filter block, or the texture complexity of the filter block can be determined based on the image region that has a mapping relationship with the filter block, thus improving the flexibility of determining the texture complexity.

[0063] In some embodiments, determining the texture complexity of a filter block based on pixel values ​​in the filter block includes any of the following: performing smooth filtering on the filter block to obtain an updated filter block; determining the texture complexity of the filter block based on the difference between the pixel values ​​of the filter block and the pixel values ​​of the updated filter block; and determining the texture complexity of the filter block using the intra-frame prediction cost and / or inter-frame prediction cost of the filter block.

[0064] Smoothing filtering can be implemented using any smoothing filtering algorithm, including but not limited to Gaussian filtering, bandpass filtering, lowpass filtering, or downsampling. Taking Gaussian filtering as an example, if the Gaussian kernel is... Figure 6 As shown, by performing Gaussian filtering on the pixel value p(i,j) of the pixel in the i-th row and j-th column of the filter block, we can obtain... and (This formula is a simplified version of the Gaussian filter formula).

[0065] For example, the pixel difference between pixels at the same position in the filter block and the updated filter block can be determined. Specifically, pixels in the filter block are referred to as first pixels, and pixels in the updated filter block are referred to as second pixels. For each first pixel in the filter block, a second pixel at the same position as the first pixel is determined from the updated filter block, and the difference between the pixel value of the first pixel and the pixel value of the corresponding second pixel is calculated to obtain the pixel difference. The average of the pixel differences can be used as the texture complexity of the filter block.

[0066] Similarly, the texture complexity of an image region can be determined using pixel values ​​within that region. Specifically, the image region can be smoothed using a filtering process to obtain an updated region, and the texture complexity can be determined based on the difference between the pixel values ​​of the updated region and the pixel values ​​of the newer region. Alternatively, the texture complexity can be determined using the intra-frame prediction cost and / or inter-frame prediction cost of the image region.

[0067] For example, texture complexity can be calculated using the following formula:

[0068]

[0069] Where N is the total number of pixels in any image block, such as a filter block or an image region, for which texture complexity needs to be calculated. n This represents the pixel value of the nth pixel in the image block. This represents updating an image block, such as updating a filter block or updating the pixel value of the nth pixel in an image region, i.e., for p n The pixel value obtained after smoothing filtering. C tex This represents the texture complexity of the image patch for which texture complexity needs to be calculated.

[0070] For example, texture complexity is positively correlated with intra-frame prediction cost and inter-frame prediction cost. Intra-frame prediction cost can be represented as `intraCost`, and inter-frame prediction cost as `interCost`. `intraCost` is the estimated "cost" of encoding a picture patch (or an entire frame) using intra-frame prediction modes; `interCost` is the estimated "cost" of encoding a picture patch (or an entire frame) using inter-frame prediction modes. For instance, the texture complexity of a picture patch can be determined by pre-calculating the encoding costs, such as the intra-frame prediction cost and / or inter-frame prediction cost, of the picture patch in a preprocessing module, such as a lookahead (pre-encoding) module. Here, lookahead is an initial step in the encoding pipeline that analyzes the input video sequence before formal encoding to understand its motion characteristics and complexity, providing guidance for subsequent encoding decisions (such as bitrate control and motion estimation).

[0071] In this embodiment, the texture complexity of the filter block can be accurately determined by smoothing the pixel differences between the filter blocks before and after filtering. Existing coding standards (such as H.266) obtain rich motion information in their precoding stage, but this information is not fully utilized in subsequent processing modules such as MCTF, which easily leads to a waste of computational resources when searching large areas. However, by using the intra-frame prediction cost and / or inter-frame prediction cost of the image region corresponding to the filter block to determine the texture complexity of the image region, the coding cost obtained in the precoding stage can be fully utilized to improve search efficiency and search accuracy.

[0072] In some embodiments, texture complexity includes luminance texture complexity and chrominance texture complexity. Determining the search bias value of the filter block based on texture complexity and / or intensity characterization value includes: determining the search bias value of the filter block based on luminance texture complexity, chrominance texture complexity and intensity characterization value.

[0073] Among them, the search deviation value is negatively correlated with the luminance texture complexity, and positively correlated with the chrominance texture complexity.

[0074] For example, the search bias value can be positively correlated with the sum of the first chroma texture complexity and the second chroma texture complexity. The product of this sum and the intensity characterization value can be calculated, and the search bias value of the filter block can be determined based on the ratio of this product to the luminance texture complexity.

[0075] In this embodiment, since the search deviation value is negatively correlated with the luminance texture complexity and positively correlated with the chrominance texture complexity, the search deviation value will decrease as the complexity of the luminance component increases and decrease as the complexity of the chrominance component decreases (flat areas are not sensitive to error feedback and are easily lost in the pyramid structure), which helps to improve the accuracy of the search deviation value.

[0076] In some embodiments, determining the search deviation value of the filter block based on texture complexity and / or intensity characterization value includes: determining a deviation factor based on the intensity characterization value, wherein the deviation factor is negatively correlated with the intensity characterization value; and determining the search deviation value of the filter block based on texture complexity and the deviation factor, wherein the search deviation value is negatively correlated with the deviation factor.

[0077] For example, the deviation factor can be calculated using the following formula:

[0078]

[0079] Where, δ scc SCC(R) represents the deviation factor, and SCC(R) represents the intensity characterization value.

[0080] For example, a computer device can calculate the search bias value of a filter block using the following formula:

[0081]

[0082] Where σ represents the search bias value of the filter block. Represents the complexity of the first chromaticity texture. Represents the complexity of the second chromaticity texture. δ represents the brightness texture complexity. scc The deviation factors are represented by k1, k2, and k3, which are preset coefficients. k1 is a positive number, while k2 and k3 can be positive or negative. Overall, the search deviation value decreases as the complexity of the luminance component of strong SCC (Screen Content Coding) content increases (the search accuracy is higher in complex SCC regions), and decreases as the complexity of the chrominance component decreases (flat regions are less sensitive to error feedback and are easily lost in the pyramid structure).

[0083] In this embodiment, since the deviation factor is negatively correlated with the intensity characterization value, and the search deviation value is negatively correlated with the deviation factor, the search deviation value is positively correlated with the intensity characterization value. Therefore, the search deviation value of the filter block with a small intensity characterization value is less than the search deviation value of the filter block with a large intensity characterization value, and the search deviation value of non-SCC content is relatively lower than that of SCC content.

[0084] In some embodiments, determining the intensity characterization value corresponding to the screen content intensity of the filter block includes: determining the pixel difference between the maximum and minimum pixel values ​​in the filter block and the pixel value variance of the filter block; determining the hash code value of the filter block and determining the hash repetition count of the hash code value; determining the entropy of the pixel values ​​in the filter block to obtain the pixel value entropy; and determining the intensity characterization value corresponding to the screen content intensity of the filter block based on the pixel difference, pixel value variance, hash repetition count and / or pixel value entropy.

[0085] The difference between the maximum and minimum pixel values ​​in the filter block is D = max - min, where max represents the maximum pixel value and min represents the minimum pixel value.

[0086] For example, the pixel value variance of a filter block refers to the variance of pixel values ​​within the filter block. The pixel value variance can be calculated using a standard variance calculation formula, or a simplified version of the formula can be used. For example, the simplified variance calculation formula is:

[0087]

[0088] Where N is the number of pixels contained in the filter block, for example, N = 64. ij Let σ be the pixel value of the pixel in the i-th row and j-th column of the filter block, and μ be the average pixel value of the filter block.2 This represents the variance of pixel values ​​in the filter block.

[0089] For example, for each video frame at different scales of the original video frame, the hash code value of each filter block (search block) in the video frame can be determined. For instance, the position information and pixel values ​​in the filter block can be hash-coded to obtain the hash code value. Thus, the hash code value is jointly determined by the position information and pixel values, with the position information indicating the position of the filter block in the video frame. Since the hash code values ​​of different filter blocks may be the same, duplicate hash code values ​​may exist within the same video frame.

[0090] For example, the hash repetition count refers to the number of times the hash code value of the filter block is repeated among the hash code values ​​of all filter blocks in the video frame to which the filter block belongs. For instance, if the filter block belongs to the original video frame, and the original video frame includes K filter blocks, then there are a total of K hash code values ​​(which may be repeated). For each of the K filter blocks, the hash repetition count can be determined based on the K hash code values.

[0091] For example, the pixel value entropy of a filter block refers to the entropy of the pixel values ​​in the filter block, which is a weighted sum of the probabilities of pixel values ​​appearing in the filter block. For instance, the formula for the pixel value entropy of a filter block is:

[0092]

[0093] Where v represents the pixel value in the filter block, p v This represents the probability of pixel value v occurring.

[0094] In this embodiment, the intensity characterization value corresponding to the screen content intensity of the filter block is determined based on the pixel difference, pixel value variance, hash repetition count and / or pixel value entropy. This allows the intensity characterization value to accurately reflect the complexity of the content in the filter block, thereby accurately reflecting the screen content intensity.

[0095] In some embodiments, determining the intensity characterization value corresponding to the screen content intensity of the filter block based on pixel difference, pixel value variance, hash repetition count, and / or pixel value entropy includes: determining the intensity characterization value corresponding to the screen content intensity of the filter block based on the number of conditions that are met among multiple conditions; wherein, the multiple conditions include pixel difference less than or equal to a first difference threshold and pixel value variance less than a variance threshold, hash repetition count greater than or equal to a count threshold, pixel value entropy less than or equal to an entropy threshold, pixel difference greater than a second difference threshold, and / or pixel difference greater than a third difference threshold.

[0096] Among them, the second difference threshold is less than the third difference threshold. "Pixel difference is less than or equal to the first difference threshold and pixel value variance is less than the variance threshold" is one condition, "hash repetition count is greater than or equal to the count threshold" is another condition, "pixel value entropy is less than or equal to the entropy threshold" is another condition, "pixel difference is greater than the second difference threshold" is another condition, and "pixel difference is greater than the third difference threshold" is another condition. Thus, these multiple conditions include at least one of the five conditions.

[0097] For example, a computer device may use the number of conditions that are met among the multiple conditions as the intensity characterization value corresponding to the screen content intensity of the filter block.

[0098] In this embodiment, the intensity characterization value is determined by multiple conditions formed by thresholds, which can quickly and accurately determine the intensity characterization value.

[0099] In some embodiments, the method further includes at least one of the following: the difference threshold includes a first difference threshold, a second difference threshold, and a third difference threshold, wherein the first difference threshold is determined based on the bit depth of the original video frame, and the second and third difference thresholds are determined based on the number of different pixel values ​​in the filter block; the method threshold, the entropy threshold, and the number threshold are determined based on the bit depth of the original video frame.

[0100] The first difference threshold can be represented by T. diff This means that the variance threshold can be represented by T. std This means that the entropy threshold can be represented by T. ent This means that the number of times threshold can be represented by T. hash The bit depth (bd) represents the number of binary bits per channel. The bit depth can be, but is not limited to, 8 bits or 10 bits. An 8-bit bit depth means that each channel is represented by 8 binary bits, and a 10-bit bit depth means that each channel is represented by 10 binary bits.

[0101] For example, the first difference threshold T diff =2 << (bd-8), variance threshold T std =4 << (bd-8), entropy threshold T ent =3 + (bd>8), threshold number T hash = 4 - (bd > 8). Where, left shift, shifting left by one bit is equivalent to multiplying by 2. For example, if bd = 10 bits, then T diff =2《2=2×2 2 =8,T std =4<<2=4×2 2 =16,T ent =3 + (10 > 8) = 3 + 1 = 4, T hash=4-(10>8)=4-1=3; For example, if bd=8bit, then T diff =2<<0=2×2 0 =2,T std =4<<2=4×2 0 =4,T ent =3 + (8 > 8) = 3 + 0 = 3, T hash =4-(8>8)=4-0=4.

[0102] For example, the number of different pixel values ​​in the filter block can be represented as k. The second difference threshold can be determined based on the first coefficient and the number of different pixel values ​​in the filter block, and the third difference threshold can be determined based on the second coefficient and the number of different pixel values ​​in the filter block. The first coefficient can be represented by T. weak This indicates that the second coefficient can be represented by T. high This indicates that the first coefficient is less than the second coefficient; for example, the first coefficient is 1 and the second coefficient is 4.

[0103] For example, the second difference threshold is positively correlated with the first coefficient, and the second difference threshold is positively correlated with the number of different pixel values ​​in the filter block. For example, the second difference threshold = T weak (k-1).

[0104] For example, the third difference threshold is positively correlated with the second coefficient, and the third difference threshold is also positively correlated with the number of different pixel values ​​in the filter block. For instance, the third difference threshold = T high (k-1).

[0105] For example, the formula for calculating the intensity characterization value can be: SCC(R)=[D≤T diff And σ 2 <T std ]+[Hash repetition count ≥ T hash ]+[Pixel value entropy≤T ent ]+[D>T weak ×(k-1)]+[D>T high ×(k-1)]. Here, [·] is an indicator function. If the condition in [·] is true, the value of [·] is 1; otherwise, the value of [·] is 0. Therefore, the value range of SCC(R) is 0 to 5.

[0106] In this embodiment, the accuracy of the intensity characterization value can be improved by precisely setting each threshold.

[0107] In some embodiments, expanding the size of the filter block according to the search deviation value to obtain the expanded size of the filter block includes: determining an expansion ratio based on the search deviation value, wherein the expansion ratio is positively correlated with the search deviation value; and expanding the size of the filter block according to the expansion ratio to obtain the expanded size of the filter block.

[0108] For example, the length and width of the filter block can be the same, and the size of the filter block can be set as needed, for example, an 8×8 image block. Of course, the length and width of the filter block can also be different. The search deviation value can be used as the expansion ratio. The expansion size is positively correlated with the expansion ratio.

[0109] For example, the length of the filter block can be expanded according to the expansion ratio to obtain the expanded length, and the width of the filter block can be expanded according to the expansion ratio to obtain the expanded width. The expanded length and the expanded width are used as the expansion size of the filter block.

[0110] For example, the expansion size of the filter block can be the same for each channel, and the expansion size for each channel can be calculated using the following formula:

[0111] S f,i =S f ×(1+σ i )

[0112] Where, σ i S refers to the search deviation value of the i-th filter block in the video frame, which is also the expansion ratio. f This represents the size of the filter block. S f,i This represents the expansion size of the i-th filter block in the video frame.

[0113] In this embodiment, since the expansion ratio is positively correlated with the search deviation value, the filter block with a larger search deviation value can obtain a larger expansion size.

[0114] In some embodiments, the expansion ratio includes a first expansion ratio of the filter block in the luminance channel and a second expansion ratio of the filter block in the chroma channel, and the expansion size includes a first expansion size of the filter block in the luminance channel and a second expansion size of the filter block in the chroma channel; determining the expansion ratio based on the search deviation value includes: determining the first expansion ratio of the filter block in the luminance channel based on the search deviation value, and determining the second expansion ratio of the filter block in the chroma channel based on the first expansion ratio and the increment ratio; wherein the first expansion size is obtained by expanding the size of the filter block according to the first expansion ratio, and the second expansion size is obtained by expanding the size of the filter block according to the second expansion ratio.

[0115] The increment ratio can be set according to actual needs; for example, the increment ratio can be 0.25, or 25%. The second expansion ratio is positively correlated with the first expansion ratio and the increment ratio. For example, the sum of the first expansion ratio and the increment ratio can be used as the second expansion ratio.

[0116] For example, the first extended size can utilize Sf,i =S f ×(1+σ i The calculation yields σ, where σ is the most significant σ. i S represents the first expansion ratio. f,i This represents the first extended size. It can be understood that in S... f,i When S is an integer, f,i As the first extended size, in S f,i When S is a decimal, it can be used for... f,i Rounding is performed to obtain the first extended size, for example, by rounding to the nearest integer.

[0117] For example, the second extended dimension can utilize S′ f,i =S f ×(1+σ i +σ c ) Calculation yielded σ c σ represents the incremental ratio. i +σ c S′ represents the second expansion ratio. f,i This represents the second extended dimension. It can be understood that in S′... f,i When S' is an integer, f,i As a second extended dimension, in S′ f,i When it is a decimal, we can use S′ f,i Rounding, for example, to the nearest integer, yields the second extended size.

[0118] In this embodiment, since the search deviation under the chroma component (chroma channel) is generally higher than that under the luminance component (luminance channel), an additional incremental ratio is allocated to the chroma component, which can further improve the accuracy of the search.

[0119] The temporal filtering method provided in this application, applied to MCTF, can adaptively expand the size of MCTF filter blocks based on texture characteristics, realize a scheme for calculating the error probability of filter blocks, i.e., the search deviation value, evaluate the prediction misalignment of the MCTF pyramid structure based on texture complexity, screen content intensity, and channel attributes, distinguish the influence of luminance and chrominance channel texture complexity on the MCTF motion estimation error probability, and implement a new screen content intensity definition method, which is used in the size expansion of MCTF filter blocks.

[0120] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0121] Based on the same inventive concept, this application also provides a time-domain filtering device for implementing the time-domain filtering method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more time-domain filtering device embodiments provided below can be found in the limitations of the time-domain filtering method described above, and will not be repeated here.

[0122] In some embodiments, such as Figure 7 As shown, a time-domain filtering device is provided, including: a block determination module 702, an information determination module 704, a deviation determination module 706, a size expansion module 708, and a search module 710, wherein:

[0123] The block determination module 702 is used to determine the filtering block corresponding to the original video frame to be filtered.

[0124] The information determination module 704 is used to determine the texture complexity of the filter block and the intensity characterization value corresponding to the screen content intensity of the filter block. The screen content intensity is used to reflect the complexity of the content in the filter block, and the texture complexity is used to reflect the complexity of the texture in the filter block.

[0125] The deviation determination module 706 is used to determine the search deviation value of the filter block based on the texture complexity and / or intensity characterization value. The search deviation value is used to reflect the magnitude of the deviation in searching for motion information according to the size of the filter block.

[0126] The size expansion module 708 is used to expand the size of the filter block according to the search deviation value to obtain the expanded size of the filter block.

[0127] The search module 710 is used to search for motion information of the filter block according to the extended size, and the results of the motion information search are used to perform temporal filtering on the original video frame.

[0128] In some embodiments, the motion estimation process of the original video frame includes multi-layer integer pixel motion estimation and multi-layer sub-pixel search. The filter block is the search block used in the integer pixel motion estimation or the search block used in the sub-pixel search. Each layer of integer pixel motion estimation uses video frames of different scales of the original video frame. The information determination module 704 is also used to determine the texture complexity of the filter block based on the pixel values ​​in the filter block. Alternatively, it can determine the image region that has a mapping relationship with the filter block from video frames of different scales, and use the texture complexity of the image region as the texture complexity of the filter block. The image region contains the filter block or the image region contains a downsampled image block of the filter block.

[0129] In some embodiments, the information determination module 704 is further configured to perform smooth filtering on the filter block to obtain an updated filter block, and determine the texture complexity of the filter block based on the difference between the pixel values ​​of the filter block and the pixel values ​​of the updated filter block; or, determine the texture complexity of the filter block using the intra-frame prediction cost and / or inter-frame prediction cost of the filter block.

[0130] In some embodiments, texture complexity includes luminance texture complexity and chrominance texture complexity. The deviation determination module 706 is further configured to determine the search deviation value of the filter block based on the luminance texture complexity, chrominance texture complexity, and intensity characterization value; wherein the search deviation value is negatively correlated with luminance texture complexity and positively correlated with chrominance texture complexity.

[0131] In some embodiments, the deviation determination module 706 is further configured to determine a deviation factor based on the intensity characterization value, wherein the deviation factor is negatively correlated with the intensity characterization value; and to determine a search deviation value of the filter block based on the texture complexity and the deviation factor, wherein the search deviation value is negatively correlated with the deviation factor.

[0132] In some embodiments, the information determination module 704 is further configured to determine the pixel difference between the maximum and minimum pixel values ​​in the filter block and the pixel value variance of the filter block; determine the hash code value of the filter block and determine the hash repetition count of the hash code value; determine the entropy of the pixel values ​​in the filter block and obtain the pixel value entropy; and determine the intensity characterization value corresponding to the screen content intensity of the filter block based on the pixel difference, pixel value variance, hash repetition count and / or pixel value entropy.

[0133] In some embodiments, the information determination module 704 is further configured to determine the intensity characterization value corresponding to the screen content intensity of the filter block based on the number of conditions that are met among a plurality of conditions; wherein the plurality of conditions include a pixel difference less than or equal to a first difference threshold and a pixel value variance less than a variance threshold, a hash repetition count greater than or equal to a count threshold, a pixel value entropy less than or equal to an entropy threshold, a pixel difference greater than a second difference threshold and / or a pixel difference greater than a third difference threshold.

[0134] In some embodiments, the difference threshold includes a first difference threshold, a second difference threshold, and a third difference threshold. The first difference threshold is determined based on the bit depth of the original video frame, and the second and third difference thresholds are determined based on the number of different pixel values ​​in the filter block. The method threshold, entropy threshold, and number threshold are determined based on the bit depth of the original video frame.

[0135] In some embodiments, the size expansion module 708 is further configured to determine an expansion ratio based on the search deviation value, wherein the expansion ratio is positively correlated with the search deviation value; and to expand the size of the filter block according to the expansion ratio to obtain the expanded size of the filter block.

[0136] In some embodiments, the expansion ratio includes a first expansion ratio of the filter block in the luminance channel and a second expansion ratio of the filter block in the chroma channel, and the expansion size includes a first expansion size of the filter block in the luminance channel and a second expansion size of the filter block in the chroma channel; the size expansion module 708 is further configured to determine the first expansion ratio of the filter block in the luminance channel based on the search deviation value, and determine the second expansion ratio of the filter block in the chroma channel based on the first expansion ratio and the increment ratio; wherein, the first expansion size is obtained by expanding the size of the filter block according to the first expansion ratio, and the second expansion size is obtained by expanding the size of the filter block according to the second expansion ratio.

[0137] Each module in the aforementioned time-domain filtering device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0138] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data involved in the time-domain filtering method. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a time-domain filtering method.

[0139] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a time-domain filtering method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0140] Those skilled in the art will understand that Figure 8 and Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the time-domain filtering method described above.

[0142] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the time-domain filtering method described above.

[0143] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the time-domain filtering method described above.

[0144] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0145] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A time-domain filtering method, characterized in that, The method includes: Determine the filter block corresponding to the original video frame to be filtered; The texture complexity of the filter block is determined, and the intensity characterization value corresponding to the screen content intensity of the filter block is determined. The screen content intensity is used to reflect the complexity of the content in the filter block, and the texture complexity is used to reflect the complexity of the texture in the filter block. The search deviation value of the filter block is determined based on the texture complexity and / or the intensity characterization value. The search deviation value is used to reflect the magnitude of the deviation in searching for motion information according to the size of the filter block. The size of the filter block is expanded based on the search deviation value to obtain the expanded size of the filter block; Motion information is searched for the filter block according to the extended size, and the result of the motion information search is used to perform temporal filtering on the original video frame.

2. The method according to claim 1, characterized in that, The motion estimation process of the original video frame includes multi-layer integer pixel motion estimation and multi-layer sub-pixel search. The filter block is the search block used in the integer pixel motion estimation or the search block used in the sub-pixel search. Each layer of integer pixel motion estimation uses video frames of different scales from the original video frame. Determining the texture complexity of the filter block includes any of the following: The texture complexity of the filter block is determined based on the pixel values ​​in the filter block; From the video frames of different scales, an image region that has a mapping relationship with the filter block is determined, and the texture complexity of the image region is used as the texture complexity of the filter block. The image region contains the filter block or the image region contains a downsampled image block of the filter block.

3. The method according to claim 2, characterized in that, Determining the texture complexity of the filter block based on the pixel values ​​in the filter block includes any one of the following: The filter block is smoothed to obtain an updated filter block. The texture complexity of the filter block is determined based on the difference between the pixel values ​​of the filter block and the pixel values ​​of the updated filter block. The texture complexity of the filter block is determined using the intra-frame prediction cost and / or inter-frame prediction cost of the filter block.

4. The method according to any one of claims 1 to 3, characterized in that, The texture complexity includes luminance texture complexity and chrominance texture complexity. Determining the search bias value of the filter block based on the texture complexity and / or the intensity characterization value includes: The search bias value of the filter block is determined based on the luminance texture complexity, the chrominance texture complexity, and the intensity characterization value. The search deviation value is negatively correlated with the luminance texture complexity and positively correlated with the chrominance texture complexity.

5. The method according to any one of claims 1 to 3, characterized in that, Determining the search bias value of the filter block based on the texture complexity and / or the intensity characterization value includes: A deviation factor is determined based on the intensity characterization value, and the deviation factor is negatively correlated with the intensity characterization value. The search deviation value of the filter block is determined based on the texture complexity and the deviation factor, and the search deviation value is negatively correlated with the deviation factor.

6. The method according to any one of claims 1 to 3, characterized in that, Determining the intensity characterization value corresponding to the screen content intensity of the filter block includes: Determine the pixel difference between the maximum and minimum pixel values ​​in the filter block and the pixel value variance of the filter block; Determine the hash code value of the filter block, and determine the hash repetition count of the hash code value; Determine the entropy of the pixel values ​​in the filter block to obtain the pixel value entropy; Based on the pixel difference, the pixel value variance, the hash repetition count, and / or pixel value entropy, the intensity characterization value corresponding to the screen content intensity of the filter block is determined.

7. The method according to claim 6, characterized in that, The step of determining the intensity characterization value corresponding to the screen content intensity of the filter block based on the pixel difference, the pixel value variance, the hash repetition count, and / or pixel value entropy includes: Based on the number of conditions that are met among multiple conditions, the intensity characterization value corresponding to the screen content intensity of the filter block is determined. The multiple conditions include the pixel difference being less than or equal to a first difference threshold and the pixel value variance being less than a variance threshold, the hash repetition count being greater than or equal to a count threshold, the pixel value entropy being less than or equal to an entropy threshold, the pixel difference being greater than a second difference threshold, and / or the pixel difference being greater than a third difference threshold.

8. The method according to claim 7, characterized in that, The method further includes at least one of the following: The difference thresholds include a first difference threshold, a second difference threshold, and a third difference threshold. The first difference threshold is determined based on the bit depth of the original video frame, and the second and third difference thresholds are determined based on the number of different pixel values ​​in the filter block. The method threshold, the entropy threshold, and the number threshold are determined based on the bit depth of the original video frame.

9. The method according to any one of claims 1 to 3, characterized in that, The step of expanding the size of the filter block according to the search deviation value to obtain the expanded size of the filter block includes: The expansion ratio is determined based on the search deviation value, and the expansion ratio is positively correlated with the search deviation value. The size of the filter block is expanded according to the expansion ratio to obtain the expanded size of the filter block.

10. The method according to claim 9, characterized in that, The expansion ratio includes a first expansion ratio of the filter block in the luminance channel and a second expansion ratio of the filter block in the chroma channel; the expansion size includes a first expansion size of the filter block in the luminance channel and a second expansion size of the filter block in the chroma channel; determining the expansion ratio based on the search deviation value includes: The first expansion ratio of the filter block in the luminance channel is determined based on the search deviation value, and the second expansion ratio of the filter block in the chroma channel is determined based on the first expansion ratio and the increment ratio. Wherein, the first expansion size is obtained by expanding the size of the filter block according to the first expansion ratio, and the second expansion size is obtained by expanding the size of the filter block according to the second expansion ratio.

11. A time-domain filtering device, characterized in that, The device includes: The block determination module is used to determine the filtering block corresponding to the original video frame to be filtered; An information determination module is used to determine the texture complexity of the filter block and the intensity characterization value corresponding to the screen content intensity of the filter block. The screen content intensity is used to reflect the complexity of the content in the filter block, and the texture complexity is used to reflect the complexity of the texture in the filter block. The deviation determination module is used to determine the search deviation value of the filter block based on the texture complexity and / or the intensity characterization value, wherein the search deviation value is used to reflect the magnitude of the deviation in searching for motion information according to the size of the filter block; A size expansion module is used to expand the size of the filter block according to the search deviation value to obtain the expanded size of the filter block; The search module is used to search for motion information in the filter block according to the extended size, and the result of the motion information search is used to perform temporal filtering on the original video frame.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images