Temporal filtering method, apparatus and computer program product in pre-processing procedure of hardware video encoder

Abstract

The application belongs to the technical field of video coding, and discloses a time domain filtering method, device and computer program product of a hardware video encoder in a preprocessing procedure. The method comprises the following steps: obtaining a current frame to be filtered; multiplexing a coarse motion estimation module, a fine optimization motion estimation module and a fractional motion estimation module in the hardware video encoder to perform down-sampling and motion search required by motion compensation time domain filtering on the current frame; multiplexing a rate-distortion optimization module in the hardware video encoder to perform interpolation and motion compensation required by the current frame; performing filtering processing on the current frame to obtain a filtered frame; and storing the filtered frame in a dynamic random access memory as an encoding input frame. The application multiplexes the encoder to perform time domain filtering preprocessing, does not need to independently design down-sampling, motion search and interpolation modules, can significantly reduce hardware area overhead, and simultaneously utilizes information such as transformation, quantization and block division in the encoding process to improve filtering quality and overall encoding efficiency.

Description

Technical Field

[0001] This invention relates to the field of video coding and hardware acceleration technology, and more specifically, to a temporal filtering method, apparatus, and computer program product for a hardware video encoder in the preprocessing process. Background Technology

[0002] Hardware video encoders are specialized devices designed for video data processing. Compared to purely software video encoders, they offer significantly higher video processing performance. In scenarios with stringent real-time requirements, such as video conferencing, live streaming, and online gaming, this type of hardware can complete video encoding and decoding tasks with higher processing speeds and lower transmission latency, while also effectively reducing device power consumption, thus balancing high performance and energy efficiency. Currently, to improve video compression efficiency, video encoding typically employs inter-frame prediction, which involves referencing frames from higher time layers to lower time layers, thereby reducing temporal redundancy. If noise is removed through preprocessing before video encoding, the efficiency of inter-frame prediction will be greatly improved.

[0003] Motion-compensated temporal filtering is a preprocessing technique that removes noise from low-temporal frames by fully utilizing relevant information between adjacent frames for motion estimation, interpolation, motion compensation, and filtering. This technique has been gradually introduced into various software video encoders. However, for hardware video encoders, there are multiple implementation paths for introducing motion-compensated temporal filtering, each facing its own limitations.

[0004] In existing technologies, there exists a type of scheme that uses a general-purpose processor to implement motion-compensated temporal filtering in software. In this type of scheme, the motion-compensated temporal filtering algorithm is executed by the central processing unit (CPU), and the filtered image data is then passed to the hardware video encoding core for encoding. Because motion-compensated temporal filtering involves a large amount of data and computation, the processing speed of the software implementation is significantly slower than that of the hardware video encoding core. To match the hardware encoding speed, it is often necessary to reduce the motion search range or choose interpolation and filtering methods with lower computational load, resulting in unsatisfactory filtering results. At the same time, the power consumption of a general-purpose processor executing the motion-compensated temporal filtering algorithm is also relatively high.

[0005] In existing technologies, there is another type of scheme that implements motion-compensated temporal filtering as an independent hardware module and connects it to a hardware video encoding core. In this type of scheme, each functional unit required for motion-compensated temporal filtering is implemented as an independent hardware module, including independent downsampling modules, motion search modules, and interpolation modules. Because these functional modules need to be designed and implemented separately, the hardware area overhead is large, increasing the overall hardware cost.

[0006] Therefore, a hardware implementation scheme for motion compensation time-domain filtering with low area overhead and sufficient input information is needed. Summary of the Invention

[0007] To address the technical problems existing in the prior art, this invention provides a temporal filtering (Motion Compensation Temporal Filter, MCTF) method in the preprocessing flow of a hardware video encoder, comprising: acquiring the current frame to be filtered; reusing the coarse motion estimation module, the fine-tuning motion estimation module, and the fractional motion estimation module in the hardware video encoder to perform downsampling and motion search required for motion compensation temporal filtering on the current frame to be filtered; reusing the rate-distortion optimization module in the hardware video encoder to perform interpolation and motion compensation required for motion compensation temporal filtering on the current frame to be filtered; performing filtering processing on the current frame to be filtered to obtain a filtered frame; and storing the filtered frame in a dynamic random access memory as an encoding input frame.

[0008] In existing technologies, when introducing motion-compensated temporal filtering in hardware video encoders, software implementations suffer from limited processing speed and high power consumption, while independent hardware modules incur significant area overhead. Furthermore, simplified independent designs utilize limited input information. This invention reuses existing coarse motion estimation, fine-tuning motion estimation, fractional motion estimation, and rate-distortion optimization modules within the hardware video encoder. These modules respectively handle the downsampling, motion search, interpolation, and motion compensation functions required for motion-compensated temporal filtering. The tools with the highest area overhead for motion-compensated temporal filtering are all included in the reused modules, thereby significantly reducing the hardware area overhead required for implementing motion-compensated temporal filtering. Simultaneously, by reusing the complete encoding modules, richer encoding information can be obtained to improve filtering performance. Thus, motion-compensated temporal filtering is implemented in hardware with low area overhead and sufficient input information.

[0009] Optionally, filtering is performed on the current frame to be filtered to obtain a filtered frame, including: acquiring the original pixels, interpolated optimal inter-frame predicted pixels, block partitioning results, and quantization parameters of the current coding block in the current frame to be filtered; performing bilateral filtering on the current frame to be filtered based on the original pixels, interpolated optimal inter-frame predicted pixels, block partitioning results, and quantization parameters to obtain filtered pixels; and storing the filtered pixels as the filtered frame in dynamic random access memory. Thus, the filtering process can fully utilize multi-dimensional coding information such as original pixels, optimal inter-frame predicted pixels, block partitioning results, and quantization parameters at the coding block level, improving the filtering effect and enhancing coding quality.

[0010] Optionally, motion prediction and rate-distortion optimization (RDE) decisions are performed on coded blocks of 8x8, 16x16, and 32x32 sizes in the current frame to be filtered. The prediction block corresponding to the optimal block size is used as the input for filtering. The transform in the RDE decision is a discrete cosine transform, a discrete sine transform, or a Hadamard transform. Therefore, by obtaining the optimal block size through motion prediction and RDE decisions with multiple block sizes, the problem of poor filtering performance caused by a fixed block size is avoided, further improving the filtering effect.

[0011] Optionally, the frame sequence is organized into image groups of four frames each, with frame types arranged according to IBII. I-frames are complete frames and do not require filtering. B-frames are bidirectional reference frames and require filtering to ensure that temporal filtering is performed once every four frames. The image group supports all random access image groups with a GOP length that is a multiple of four. Frame type determination is performed before the encoding step. If the current frame is a B-frame, the filtered frame is retrieved from the dynamic random access memory as the encoding input frame; otherwise, the original frame is retrieved from the original image data. Thus, the filtering semantics of I-frames and B-frames are clearly defined through the IBII image group architecture, ensuring that temporal filtering is performed once every four frames and aligning with all random access image groups with a GOP length that is a multiple of four, guaranteeing frame type alignment between the temporal filtering and encoding steps.

[0012] Optionally, when the temporal filtering method starts, two invalid frames are counted first, establishing an offset relationship of two frames between the frame index of the frame processed by the temporal filtering method and the frame index of the encoding step. When the encoding step inserts an intraframe at frame index N, a complete frame is inserted at the corresponding position of the frame processed by the temporal filtering method. The corresponding position is determined to be the position of frame index N+2 of the frame processed by the temporal filtering method based on the offset relationship. Thus, by counting two invalid frames, an index offset relationship is established between the frames processed by the temporal filtering method and the frames of the encoding step. When the encoding step inserts an intraframe to refresh the image group, the temporal filtering method can insert a complete frame at the corresponding position to achieve alignment, ensuring that a new IBII image group is connected after the image group is refreshed, thus improving the stability and robustness of the frame structure.

[0013] Optionally, at least one frame of shared space is reserved in the dynamic random access memory. When the time-domain filtering method completes filtering and stores the filtered frame in the shared space, the time-domain filtering method raises the filtering completion signal and pauses filtering. After receiving the filtering completion signal, the encoding step lowers the signal and starts working. When the filtered frame is encoded, the encoding completion signal is raised and the work stops. After reading the encoding completion signal, the time-domain filtering method lowers the signal and starts filtering the next frame. Thus, through the dual-signal handshake mechanism of the filtering completion signal and the encoding completion signal, synchronization between the time-domain filtering method and the encoding step is achieved, ensuring stable transmission of filtered frame data in the shared space.

[0014] Optionally, the original pixels of the current frame to be filtered are obtained; a discrete cosine transform or discrete sine transform is first performed on the original pixels, followed by quantization, inverse quantization, and inverse transform to obtain reconstructed pixels; a rate-distortion value is calculated based on the difference between the original pixels and the reconstructed pixels; the rate-distortion value is used to optimize the filtering intensity of the temporal filtering method to determine the filtering intensity of the filtering process. In the prior art, conventional motion-compensated temporal filtering, due to the lack of a transform quantization module, can only obtain the rate-distortion value by directly subtracting the predicted pixels from the original pixels, which cannot reflect the impact of transform and quantization distortion on the filtering effect. In this invention, because a complete rate-distortion optimization module is reused, the rate-distortion value is calculated based on the original pixels minus the reconstructed pixels after transform, quantization, inverse quantization, and inverse transform, comprehensively reflecting the impact of transform and quantization distortion. This allows for a better selection of the filtering intensity for each coding block, improving the filtering effect.

[0015] Optionally, when rate-distortion optimization in the encoding step is completed and the next frame is a filtered frame, the input data sources of the coarse motion estimation module, the fine-tuned motion estimation module, and the fractional-level motion estimation module are pre-switched to the motion compensation temporal filtering reference frame. Thus, by pre-switching the module input data sources in advance when rate-distortion optimization in the encoding step is completed and the next frame is a filtered frame, pipeline bubbles are eliminated, time slot conflicts during the filtering frame's wait for module release are avoided, and hardware multiplexing efficiency is improved.

[0016] Optionally, the rate-distortion value calculated by the rate-distortion optimization module and the quantization parameters of the encoding step are obtained; based on the rate-distortion value and the quantization parameters, bilateral filtering parameters are calculated. These bilateral filtering parameters include spatial domain filtering parameters and tolerance domain filtering parameters, which are calculated using closed-form functions based on the rate-distortion value and the quantization parameters. In existing technologies, bilateral filtering parameters are typically determined using fixed settings or offline table lookup methods, resulting in poor content adaptability of the filtering strength. In this invention, the bilateral filtering parameters are calculated using closed-form functions based on the rate-distortion value and the quantization parameters, enabling the filtering strength to be adaptively determined according to the rate-distortion state of the current frame, thus improving the filtering effect and content adaptability.

[0017] Optionally, the mean and variance of the statistical rate distortion values ​​are calculated; based on the mean and variance of the rate distortion values, the bilateral filter parameters are calculated. Thus, by using the mean and variance of the statistical rate distortion values, the problem of single-point rate distortion values ​​being affected by outliers is suppressed, making the calculation of the bilateral filter parameters more robust.

[0018] Optionally, multiplication and division are performed on the rate-distortion value and the quantization parameter, and the square root result is obtained by querying a pre-stored square root lookup table. The multiplication and division are 16-bit fixed-point operations, and the output of the bilateral filter parameters is a 10-bit fixed-point number. Thus, hardware calculation of the bilateral filter parameters is realized through 16-bit fixed-point operations and a square root lookup table, avoiding the high implementation cost of floating-point operations, making analytical mapping feasible in hardware, and reducing implementation cost.

[0019] Optionally, the frame type information is physically encoded in the address space of a dynamic random access memory (DRAM). The DRAM includes a first storage area and a second storage area. The first storage area stores the original frame, and the second storage area stores the filtered frame. The data retrieval source for the encoding step is automatically routed by the high-order bits of the address. When the lower two bits of the current frame index are 00, the high-order bits of the address point to the second storage area; otherwise, they point to the first storage area. In the prior art, conventional single-buffered DRAMs do not distinguish between original frames and filtered frames, and data retrieval requires additional data retrieval control logic to distinguish the frame type, leading to increased control complexity and area overhead. In this invention, the frame type information is physically encoded in the address space, and the data retrieval source is automatically routed to the corresponding storage area by the high-order bits of the address, eliminating the need for additional data retrieval control logic and reducing control complexity and area overhead.

[0020] Optionally, N credit tokens are configured; one credit token is consumed for each frame filtering completed; one credit token is returned for each frame consumed in the encoding step; when the credit token count is 0, temporal filtering is paused; when the credit token count is N, the encoding step is paused; the number of credit tokens N=2, the credit token count bit width is 2 bits; the frame counter bit width is 16 bits. In the prior art, the temporal filtering process and the encoding step share a dynamic random access memory. Without a flow control mechanism, a mismatch between the data supply rate on the filtering side and the consumption rate on the encoding side can lead to buffer overflow or data conflict. In this invention, a credit token mechanism is used to achieve bidirectional flow control between the temporal filtering and encoding steps, avoiding buffer overflow and data conflict, and bidirectional pauses do not require additional handshake signals.

[0021] Optionally, the quantization error energy and frequency domain distribution corresponding to the current frame to be filtered are obtained; the filtering intensity of the filtering process is determined based on the quantization error energy and frequency domain distribution; the filtering process is performed on the current frame to be filtered using the filtering intensity; the quantization error energy bit width is 16 bits. In the prior art, the filtering intensity of motion-compensated temporal filtering is fixed or coarsely adjusted. Areas with large quantization errors are under-filtered, resulting in residual noise, while areas with small quantization errors are filtered, leading to blurred texture details. In this invention, the filtering intensity is adaptively determined jointly based on the quantization error energy and frequency domain distribution. Strong filtering is applied to flat areas with large quantization errors to remove noise, while weak filtering or bypassing is used to preserve texture details in areas with small quantization errors or rich textures, thereby improving the filtering effect.

[0022] Optionally, the coarse motion estimation module, the refined optimized motion estimation module, and the fractional motion estimation module can be replaced by an existing inter-frame prediction module in the hardware video encoder to perform the downsampling and motion search.

[0023] Optionally, the filtering process may employ filtering methods other than bilateral filtering, including but not limited to Gaussian filtering, median filtering, or Wiener filtering, to perform filtering processing on the current frame to be filtered.

[0024] This invention also provides a hardware video encoder, comprising: a coarse motion estimation module, a fine-tuned optimized motion estimation module, and a fractional-level motion estimation module, used for downsampling and motion search required for performing motion-compensated temporal filtering on the current frame to be filtered; a rate-distortion optimization module, used for interpolation and motion compensation required for performing motion-compensated temporal filtering on the current frame to be filtered; a filtering processing module, used for performing filtering processing on the current frame to be filtered to obtain a filtered frame; and a dynamic random access memory, used for storing the filtered frame as an input frame for encoding.

[0025] In existing technologies, when introducing motion-compensated temporal filtering in hardware video encoders, software implementations suffer from limited processing speed and high power consumption, while independent hardware modules incur significant area overhead and rely on external memory and independent computing units, resulting in high data transfer costs and low module utilization. This invention reuses existing coarse motion estimation, fine-tuning motion estimation, fractional motion estimation, and rate-distortion optimization modules in the hardware video encoder to handle the downsampling, motion search, interpolation, and motion compensation functions required for motion-compensated temporal filtering, respectively. Only a new filtering module is needed to achieve motion-compensated temporal filtering, thereby reducing data transfer overhead, improving module reuse, and significantly reducing hardware area overhead.

[0026] Optionally, the filtering module is further configured to: obtain the original pixels, the interpolated optimal inter-frame predicted pixels, the block partitioning result, and the quantization parameters of the current coding block in the current frame to be filtered; perform bilateral filtering on the current frame to be filtered based on the original pixels, the interpolated optimal inter-frame predicted pixels, the block partitioning result, and the quantization parameters to obtain the filtered pixels; and store the filtered pixels as the filtered frame in the dynamic random access memory. Thus, the filtering module can fully utilize the multi-dimensional coding information at the coding block level to perform bilateral filtering, improving the matching degree between the filtered frame and the coding quality.

[0027] Optionally, the dynamic random access memory (DRAM) includes a first storage area and a second storage area. The first storage area stores the original frames, and the second storage area stores the filtered frames. Frame type information is physically encoded in the address space of the DRAM. The data retrieval source for the encoding step is automatically routed by the high-order bits of the address. When the lower two bits of the current frame index are 00, the high-order bits of the address point to the second storage area; otherwise, they point to the first storage area. Thus, the frame type information is physically encoded in the address space, and the data retrieval source is automatically routed to the corresponding storage area by the high-order bits of the address, simplifying the routing mechanism and reducing management overhead.

[0028] Optionally, a credit token counting unit is also included. This unit is used to: configure N credit tokens; consume 1 credit token for each frame filtering completed; return 1 credit token for each frame consumed in the encoding step; pause temporal filtering when the credit token count is 0; pause the encoding step when the credit token count is N; the number of credit tokens N=2, the credit token counting bit width is 2 bits, and the frame counter bit width is 16 bits. Thus, bidirectional flow control between temporal filtering and encoding steps is achieved through the credit token counting unit, avoiding data backlog or module idleness caused by rate mismatch.

[0029] Furthermore, the present invention also provides a computer program product or a computer-readable storage medium having a computer program. When executed by a processor, the computer program implements the steps of the above-described time-domain filtering method.

[0030] In summary, this invention reuses existing coarse motion estimation, fine-tuned motion estimation, fractional motion estimation, and rate-distortion optimization modules in a hardware video encoder to achieve the downsampling, motion search, interpolation, and motion compensation functions required for motion-compensated temporal filtering. Only a new filtering module is needed to complete the motion-compensated temporal filtering, significantly reducing hardware area overhead. Simultaneously, by reusing the complete encoding module, rich encoding information such as block partitioning results, quantization parameters, and rate-distortion values ​​can be obtained. Furthermore, filtering performance is improved through rate-distortion value-driven adaptive calculation of bilateral filtering parameters and joint determination of filtering strength based on quantization error energy and frequency domain distribution. In addition, mechanisms such as IBII image group architecture, frame index offset alignment, dual-signal handshake, address space physical coding, and credit token flow control ensure efficient and stable interaction and data transmission between the temporal filtering process and encoding steps. Therefore, this invention achieves hardware-based motion-compensated temporal filtering with low area overhead and sufficient input information, effectively solving the technical problems existing in the prior art. Attached Figure Description

[0031] Figure 1 This is an overall flowchart of the motion compensation time-domain filtering method according to an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram of the internal structure of the filtering module and the bilateral filtering process according to an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of the code block motion prediction and rate-distortion optimization decision according to an embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram illustrating the IBII image group architecture and the collaboration between temporal filtering and encoding steps in an embodiment of the present invention.

[0035] Figure 5 This is a schematic diagram of the rate-distortion value calculation process according to an embodiment of the present invention.

[0036] Figure 6 This is a timing diagram of the pre-switching mechanism according to an embodiment of the present invention.

[0037] Figure 7 This is a schematic diagram of the bilateral filter parameter parsing and mapping process according to an embodiment of the present invention.

[0038] Figure 8 This is a schematic diagram of the address routing for the dual-storage area of ​​the dynamic random access memory according to an embodiment of the present invention.

[0039] Figure 9 This is a schematic diagram of the timing of bidirectional flow control using a credit token according to an embodiment of the present invention.

[0040] Figure 10 This is a schematic diagram illustrating the joint decision-making of quantization error energy and frequency domain distribution in an embodiment of the present invention. Detailed Implementation

[0041] First Implementation Method To address the problems of high computational complexity and large hardware resource consumption in the preprocessing workflow of hardware video encoders in the prior art, the first embodiment of the present invention proposes a temporal filtering method for hardware video encoders in the preprocessing workflow. Figure 1 This paper demonstrates the overall process of a motion-compensated temporal filtering method, including key steps such as acquiring the current frame to be filtered, performing downsampling and motion search within the multiplexed encoder's internal module, performing interpolation and motion compensation via the multiplexing rate-distortion optimization module, performing filtering to obtain the filtered frame, and storing the filtered frame in dynamic random access memory (DRAM) as the encoding input frame. This method is executed in the preprocessing flow of the hardware video encoder, which occurs before the encoding flow begins. This preprocessing flow is used to perform temporal noise reduction on the input video sequence, providing cleaner input frames for subsequent encoding steps. The current frame to be filtered is the raw image data received by the hardware video encoder, organized in luminance and chrominance component formats with a bit depth of 8 bits or 10 bits. Each pixel component is stored in memory with a fixed bit width. During the initialization sequence before the encoding flow begins, the control logic of the preprocessing flow receives the current frame data from the external bus interface, temporarily stores it in the input frame buffer area of ​​the DRAM, and generates a frame start flag and frame size parameters for subsequent processing modules to read frame by frame.

[0042] Specifically, in practical use, the hardware video encoder reuses its internal coarse motion estimation module, fine-grained motion estimation module, and fractional-level motion estimation module to perform downsampling and motion search required for motion compensation temporal filtering on the current frame to be filtered. The coarse motion estimation module takes the current frame and a reference frame as input and performs a coarse-grained motion search on blocks of a set size with integer pixel precision to obtain initial motion vector values. The fine-grained motion estimation module takes the initial motion vector values ​​output by the coarse motion estimation module as a starting point and performs a fine-grained search with integer pixel precision within a smaller search range to correct the motion vectors. The fractional-level motion estimation module performs a fractional-pixel precision search on the refined motion vectors, calculating the matching cost between half-pixel and quarter-pixel positions through interpolation to determine the final motion vector. These three modules work in series, with the output motion vector of the preceding module serving as the search starting point for the subsequent module, forming a coarse-to-fine motion search pipeline. The downsampling operation is completed by the sampling circuit within the coarse motion estimation module. This circuit extracts pixels from the input frame at a ratio of half horizontally and half vertically, generating a low-resolution image for coarse-grained searching to reduce the computational load. The reference frame data required for motion search is read from the reference frame area in dynamic random access memory, and the intermediate motion vectors and matching costs generated during the search process are temporarily stored in the register arrays inside each module.

[0043] The hardware video encoder reuses its internal rate-distortion optimization module to perform interpolation and motion compensation required for motion-compensated temporal filtering on the current frame to be filtered. The rate-distortion optimization module receives the final motion vector output by the fractional motion estimation module, obtains the corresponding pixel block from the reference frame based on this motion vector, calculates the pixel value at the fractional pixel position using an interpolation filter, and generates a predicted pixel block. Subsequently, the rate-distortion optimization module compares the predicted pixel block with the original pixel block of the current frame to be filtered pixel-by-pixel, calculates the residual, and decides on multiple coding modes based on the rate-distortion cost function. It selects the mode with the lowest rate-distortion cost as the optimal mode and outputs the predicted pixel block under the optimal mode as the motion compensation result. This motion compensation result, along with the original pixel data of the current frame to be filtered, is sent to the filtering module. The rate-distortion optimization module internally includes an interpolation filter bank, a cost calculation unit, and a mode decision unit. The interpolation filter bank uses a multi-tap finite impulse response filter to implement fractional pixel interpolation. The cost calculation unit calculates the matching cost based on the absolute difference and / or squared error. The mode decision unit compares the costs of each mode and outputs the optimal result.

[0044] The filtering module performs filtering on the current frame to be filtered, obtaining a filtered frame. In this embodiment, filtering can employ bilateral filtering or similar methods. Using the original pixels and motion-compensated predicted pixels as input, it calculates filtering weights based on pixel value differences and spatial distances, and performs a weighted average on the original pixels, preserving image edge details while removing noise. The filtering module internally includes a weight calculation unit and a weighted accumulation unit. The weight calculation unit generates filtering coefficients based on pixel value differences and spatial distances. The weighted accumulation unit sums the original pixel values ​​and predicted pixel values ​​according to the filtering coefficients, outputting the filtered pixel values. The filtering module processes all pixels of the current frame block by block, writing the processed pixels into the output buffer in raster scan order.

[0045] The filtered frames are stored in Dynamic Random Access Memory (DRAM) as encoding input frames. The filtered frames output by the filtering module are written to the filtered frame storage area in DRAM via the write data bus. This area is allocated by the control logic during preprocessing initialization and is independent of the input frame buffer area and reference frame area. After all filtered frames have been written, the control logic updates the filtered frame ready flag. Upon detecting this flag, the encoding step reads the filtered frames from DRAM as encoding input frames. For address management of the filtered frame storage area, this embodiment uses a conventional linear address mapping configuration to ensure process integrity. In case of abnormal situations, if the filtering module detects incomplete input frame data, it outputs a default pixel value and sets an error flag. The control logic determines whether to retransmit or skip the current frame based on the error flag.

[0046] In the above process, the coarse motion estimation module, the refined motion estimation module, the fractional motion estimation module, and the rate-distortion optimization module are all existing functional modules in the hardware video encoder. The motion compensation temporal filtering process directly reuses these modules, eliminating the need for additional independent downsampling circuits, motion search circuits, and interpolation circuits. The filtering module is a newly added lightweight module, containing only weight calculation and weighted accumulation logic, with a hardware area overhead far less than that of the motion estimation and interpolation circuits in the independent implementation scheme. Therefore, this implementation method achieves motion compensation temporal filtering function in the preprocessing flow of the hardware video encoder, significantly reducing hardware area overhead while ensuring filtering accuracy and improving encoding efficiency.

[0047] Second Implementation Method In the first embodiment, by reusing the coarse motion estimation module, fine-tuned motion estimation module, fractional motion estimation module, and rate-distortion optimization module within the hardware video encoder, and adding a filtering module, a basic motion compensation temporal filtering scheme with low hardware area overhead is achieved, providing temporally denoised input frames for the encoding step. The inventors of this invention have discovered that the input information and filtering method used in the filtering process directly affect the quality of the filtered frames, and thus the overall encoding efficiency of the encoder. The second embodiment of this invention further improves the filtering process based on the first embodiment. It expands the input of the filtering module to four types of information: the original pixels of the current coding block, the interpolated optimal inter-frame predicted pixels, the block partitioning result, and the quantization parameters. A bilateral filtering method is then used to jointly process these four types of information to obtain higher-quality filtered frames. Figure 2 The internal structure of the filtering module and a schematic diagram of the bilateral filtering process are shown, including the acquisition channels for four types of input information, the weight calculation unit, the weighted accumulation unit, and the output channel of the filtered pixels.

[0048] In practical use, the filtering module triggers filtering processing on the current coding block after the rate-distortion optimization module completes motion compensation and outputs the optimal inter-frame prediction pixels after interpolation. The triggering timing is uniformly controlled by the control logic of the preprocessing flow. After detecting the mode decision completion signal of the rate-distortion optimization module, the control logic sends a filtering start pulse to the filtering module, which then begins to read the four types of input information of the current coding block. The filtering module processes the current frame to be filtered block by block, and the processing cycle of each coding block is synchronized with the output rhythm of the rate-distortion optimization module, ensuring that the optimal inter-frame prediction pixels of the current coding block enter the filtering process immediately after generation, forming a pipeline-style block-level processing.

[0049] The four types of input information acquired by the filtering module have clearly defined data formats and sources. The raw pixels are the unfiltered pixel values ​​of the current coded block in the current frame to be filtered. They are read in blocks from the input frame buffer area of ​​the dynamic random access memory, with each pixel component stored with a fixed bit width, and the luminance and chrominance components organized separately. The interpolated optimal inter-frame predicted pixels are the motion compensation results output by the rate-distortion optimization module after completing fractional pixel interpolation and mode decision. This result is sent directly to the filtering module via the internal bus in the form of predicted pixel blocks, without needing external memory. The block partitioning result is output by the encoder's block partitioning decision, indicating the position and size range of the current coded block in the frame. The filtering module only uses this result as one of the filtering inputs to determine the coverage of the filtering window and the selection method of the pixel neighborhood. The quantization parameter is the quantization step size parameter used in the encoding step, provided by the encoder's quantization control unit, reflecting the encoding compression strength of the current coded block. The four types of input information are temporarily stored within the filtering module through an input register array. Once all four types of information are ready, the filtering module starts bilateral filtering calculation.

[0050] The core operation of bilateral filtering is based on the joint weighting principle of pixel value difference and spatial distance. For each pixel to be filtered in the current coding block, the filtering module selects a set of pixels in its neighborhood to participate in the weighting calculation. This set of pixels includes both the original pixels and the best inter-frame predicted pixels after interpolation. The weighting calculation unit calculates two-dimensional weighting factors for each pixel in the neighborhood: the spatial distance weight reflects the geometric proximity of the neighboring pixels to the center pixel, with a larger weight for closer pixels; the pixel value difference weight reflects the similarity of the neighboring pixels to the center pixel in terms of pixel value, with a larger weight for smaller differences. The two-dimensional weighting factors are multiplied to obtain the comprehensive weight. The weighted accumulation unit sums the pixel values ​​of each pixel in the neighborhood according to the comprehensive weight and normalizes by dividing by the total weight to obtain the filtered pixel value of the pixel to be filtered. This principle of bilateral filtering can preserve image edge details while removing noise because the pixel value difference on both sides of the edge is larger, and the corresponding pixel value difference weight is smaller, thus preserving edge information. The block partitioning result is used to limit the selection range of neighboring pixels, and the quantization parameter is used to adjust the sensitivity of the pixel value difference weight, so that the filtering intensity is adapted to the coding compression intensity. The above bilateral filtering weight calculation is a general description of the technology known in the art. This embodiment adopts a conventional bilateral filtering configuration to ensure the integrity of the process.

[0051] Intermediate results during the filtering process are temporarily stored in the register array within the filtering module. The comprehensive weights calculated by the weight calculation unit are temporarily stored in the weight register group, and the partial sums calculated by the weighted accumulation unit are temporarily stored in the accumulation register group. After the filtered pixel value of each pixel is calculated, it is immediately written to the output row buffer inside the filtering module. The output row buffer is organized according to the raster scan order. After a row of pixels has been processed, the filtered pixels are written to the filter frame storage area in the dynamic random access memory via the write data bus. This filter frame storage area is allocated by the control logic during the preprocessing initialization and is independent of the input frame buffer area and the reference frame area.

[0052] After filtering, the pixels are stored as a filtered frame in the dynamic random access memory (DRAM) and then read by the encoding step as the encoding input frame. Once the filtering module has processed all the encoded blocks of the current frame, a complete filtered frame is formed in the filtered frame storage area. When the control logic detects that the entire filtered frame has been written, it updates the filtered frame ready flag. Upon detecting this flag, the encoding step reads the filtered frame from the DRAM as the encoding input frame. In case of abnormal situations, if the filtering module detects incomplete data or synchronization errors while reading the four types of input information, it outputs a default pixel value and sets the error flag. The control logic then determines whether to retransmit or skip the current encoded block based on the error flag to ensure the integrity of the filtering process.

[0053] In the above process, the filtering module performs bilateral filtering by simultaneously utilizing four types of information: the original pixels, the optimal inter-frame prediction pixels after interpolation, the block partitioning results, and the quantization parameters. This ensures that the filtering process considers both the prediction information from temporal motion compensation and the block partitioning structure and quantization characteristics of the encoder. The filtered pixels can more accurately reflect the true image content of the current coding block. Therefore, this implementation method effectively improves the quality of the filtered frames while maintaining low hardware area overhead, thereby enhancing the coding quality.

[0054] Third Implementation Method In the second embodiment, the filtering module performs bilateral filtering using four types of information: the original pixels, the optimal inter-frame prediction pixels after interpolation, the block partitioning result, and the quantization parameters, effectively improving the quality of the filtered frames. The inventors of this invention discovered that the quality of the block partitioning result directly determines the quality of the input information for filtering, and the block partitioning result depends on the block size and transformation method used in motion prediction and rate-distortion optimization decision-making. The third embodiment of this invention, based on the second embodiment, further improves the motion prediction and rate-distortion optimization decision-making process, enabling all three sizes of coded blocks to participate in motion prediction and rate-distortion optimization decision-making, and using the prediction block corresponding to the optimal block partitioning as the input for filtering. Figure 3The diagram illustrates the motion prediction and rate-distortion optimization decision for coded blocks, including motion prediction channels for three block sizes, rate-distortion cost comparison units, and the optimal block partitioning output path.

[0055] In practical applications, motion prediction and rate-distortion optimization decisions are jointly executed by the coarse motion estimation module, fine-tuning motion estimation module, fractional-level motion estimation module, and rate-distortion optimization module, all reused within the hardware video encoder. The triggering timing is uniformly controlled by the preprocessing control logic. Upon detecting the start signal for the current frame's coded block partitioning, this logic sends a motion prediction start pulse to the coarse motion estimation module, which then begins performing an integer-pixel precision motion search on the current coded block. After completing the search, the coarse motion estimation module passes the motion vector candidate set to the fine-tuning motion estimation module. The fine-tuning module performs sub-pixel precision motion vector refinement based on the candidate set. Subsequently, the fractional-level motion estimation module performs fractional-pixel interpolation on the refined motion vectors to generate the interpolated predicted pixels. Upon receiving the outputs from the aforementioned modules, the rate-distortion optimization module performs rate-distortion optimization decisions on the candidate prediction results for each block size, calculates the corresponding rate-distortion cost, and outputs the optimal block partitioning result. The above motion prediction and rate-distortion optimization decision process is a general description of the technology known in the art. This embodiment adopts conventional motion estimation and rate-distortion optimization configuration to ensure the integrity of the process.

[0056] The current frame to be filtered is divided into three coded blocks of different sizes: 8x8, 16x16, and 32x32 during the block partitioning stage. Data for these three block sizes is read block by block from the input frame buffer area in the dynamic random access memory. Each coded block contains a luma component and a chroma component, each stored with a fixed bit width. The three coded blocks enter the motion prediction pipeline sequentially in time, with the 32x32 block entering first, followed by the 16x16 block, and finally the 8x8 block, forming a block partitioning hierarchy from largest to smallest. Each coded block, upon entering the motion prediction pipeline, carries its block position coordinates, block size identifier, and pixel data. This information is transmitted in parallel via the internal bus to the coarse motion estimation module, the fine-tuning motion estimation module, and the fractional motion estimation module.

[0057] All three coded block sizes undergo a complete motion prediction and rate-distortion optimization decision process. The 32x32 coded block has the largest motion vector search range, covering a large search window centered on its position in the current frame to capture a wide range of motion displacements; the 16x16 coded block has the second largest motion vector search range, with a search window size one-quarter that of the 32x32 coded block; and the 8x8 coded block has the smallest motion vector search range, with a search window size one-quarter that of the 16x16 coded block. After motion prediction, each of the three coded block sizes yields a set of candidate prediction blocks and their corresponding motion vectors. The rate-distortion optimization module calculates the rate-distortion cost for each candidate prediction block, which comprehensively reflects the trade-off between the coding bit rate and reconstruction distortion. The rate-distortion optimization module compares the rate-distortion costs of the three coded block sizes and selects the block partitioning method with the lowest rate-distortion cost as the optimal block partitioning. The prediction block corresponding to this optimal block partitioning is the input prediction block for motion compensation temporal filtering. The transform used in the rate-distortion optimization decision process is one of the following: discrete cosine transform, discrete sine transform, or Hadamard transform. The Discrete Cosine Transform (DCT) possesses excellent energy concentration properties, concentrating image energy onto a few low-frequency coefficients, making it suitable for most natural image content. The Discrete Sine Transform (DST) has good adaptability to intra-frame prediction residuals, effectively removing correlations from them. The Hadamard Transform (HCT) has the lowest computational complexity, involving only addition and subtraction operations, making it suitable for scenarios sensitive to computational latency. The choice of the three transform methods is determined by the rate-distortion optimization module based on the image content characteristics of the current coding block. If the current coding block is a flat region, the Hadamard Transform is preferred to reduce computational overhead; if the current coding block is a textured region, either the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST) is preferred to improve transform efficiency.

[0058] Intermediate data generated during motion prediction for the three sized coding blocks are temporarily stored in register arrays within each motion estimation module. The candidate motion vector set from the coarse motion estimation module is temporarily stored in the candidate vector register group; the refinement result from the fine-tuning optimization motion estimation module is temporarily stored in the refinement vector register group; and the interpolated pixels from the fractional motion estimation module are temporarily stored in the interpolated pixel buffer. The rate-distortion cost calculated by the rate-distortion optimization module for each coding block size is temporarily stored in the cost register group within the rate-distortion optimization module. This register group contains three independent storage areas, corresponding to the cost storage for 8x8, 16x16, and 32x32 block sizes, respectively. The optimal block partitioning result is temporarily stored in the block partitioning result register, to be read by the filtering module.

[0059] After determining the optimal block partition, the rate-distortion optimization module directly sends the predicted block corresponding to the optimal block partition to the filtering module via the internal bus, simultaneously transmitting the block partition result. Upon receiving the predicted block corresponding to the optimal block partition, the filtering module uses it as the input of the interpolated optimal inter-frame predicted pixel. This, along with the original pixel, the block partition result, and the quantization parameters, constitutes the four types of input information for bilateral filtering, performing the same bilateral filtering process as in the second implementation method. The block partition result is used to determine the coverage area of ​​the filtering window and the selection method of the pixel neighborhood, ensuring consistency between the filtering process and the partition structure of the coding block.

[0060] If incomplete data or synchronization errors are detected during motion prediction for any of the three coded block sizes, the rate-distortion optimization module skips the cost comparison for that size and uses the best result from the remaining sizes as the block partitioning output. If all three sizes are abnormal, the rate-distortion optimization module outputs the default 16x16 block partitioning result and sets an error flag. The control logic determines whether to retransmit or skip the current coded block based on the error flag to ensure the integrity of the filtering process.

[0061] In the above process, motion prediction and rate-distortion optimization decisions are performed on coded blocks of three sizes: 8x8, 16x16, and 32x32. The block with the lowest rate-distortion cost is selected as the corresponding prediction block for filtering input. The filtering module obtains the prediction information that best matches the current frame content. Motion prediction for the three block sizes covers the complete motion range from large-scale motion to small-scale details, and the selection of the optimal block partition ensures the matching accuracy between the prediction block and the original content. Therefore, this implementation effectively improves the quality of the filtering input information while maintaining low hardware area overhead, thereby improving the filtering effect.

[0062] Fourth Implementation Method In the third embodiment, all three sizes of coding blocks participate in motion prediction and rate-distortion optimization decision-making. The optimal block is used as the corresponding prediction block for filtering input, further improving the quality of the input information processed by filtering. The inventors of this invention discovered that motion compensation temporal filtering and coding steps are two parallel processing flows. The frame structure and data interaction method between them directly affect the stability and universality of the filtering process. If the frame type arrangement is improper or the data transmission lacks a synchronization mechanism, the filtering process will be difficult to work collaboratively with coding processes of different configurations. The fourth embodiment of this invention, based on the aforementioned embodiments, further improves the process collaboration mechanism between the temporal filtering method and the coding steps. It introduces three interconnected mechanisms: IBII image group architecture, invalid frame offset alignment, and shared space dual-signal handshake, enabling the filtering process to be stably embedded in the frame sequence of the coding process. Figure 4The diagram illustrates the IBII image group architecture and the collaborative temporal filtering / coding process, including the arrangement of frame types within the image group, the data flow between the temporal filtering method and the coding steps, and the timing of signal interaction in the shared space.

[0063] After startup, the temporal filtering method first organizes the input frame sequence into image groups of four frames each, with frame types within each group arranged according to IBII. I-frames are complete frames, meaning the current frame is not filtered and directly participates in the encoding step as a raw frame. B-frames are bidirectional reference frames, meaning the current frame needs to be filtered and are the filtering objects in the temporal filtering method. Since the temporal filtering algorithm inherently allows lower-time-layer frames to reference surrounding higher-time-layer frames, lower-time-layer frames in the encoding step are designated as B-frames in the temporal filtering method, while surrounding higher-time-layer frames are the unfiltered I-frames. This image group architecture can serve as a general architecture compatible with all random access image groups with GOP lengths that are multiples of 4, performing temporal filtering once every four frames to form a stable filtering rhythm. Frame type determination is executed by the control logic of the preprocessing flow, triggered before the start of encoding for each frame. The control logic reads the frame index of the current frame and performs a modulo-4 operation on the frame index. If the remainder is 1, the current frame is determined to be a B-frame, and the temporal filtering method retrieves the corresponding filtered frame from the dynamic random access memory (DRAM) as the encoding input frame. If the remainder is 0, 2, or 3, the current frame is determined to be an I-frame, and the encoding step directly retrieves the original frame from the original image data. The data path for frame type determination is as follows: the B-frame path reads the filtered image data from the filtered frame buffer area in the DRAM, and the I-frame path directly reads the original pixel data from the original image data bus. The two paths converge at the input selector in the encoding step, and the selection direction of the selector is controlled by the frame type determination result. For abnormal cases in frame type determination, this embodiment uses a conventional default configuration. If the frame type determination result is inconsistent with the image group arrangement, the control logic re-executes the determination based on the image group arrangement to ensure the integrity of the process.

[0064] When the temporal filtering method starts, it first counts two invalid frames, establishing an offset relationship of two frames between the frame index of the temporal filtering process and the frame index of the encoding step. Invalid frames are designated as X frames and do not participate in filtering; they are only used to advance the frame type determination of the image group. The timing sequence for invalid frame counting is as follows: After receiving the start signal from the encoding step, the control logic of the temporal filtering method first generates two frame index placeholders for invalid frames. Subsequently, the temporal filtering method enters the regular image group stage. Frame types within the group are arranged according to IBII, and frames with a frame index modulo 4 equal to 1 are considered low-temporal layer frames and are processed as B frames in the temporal filtering method. These frames are all low-temporal layer frames in random access image groups where the GOP length is a multiple of 4, and temporal filtering is performed once every four frames. The offset relationship is established as follows: the frame index of the temporal filtering method is equal to the frame index of the encoding step plus 2; that is, the frame index N of the encoding step corresponds to the frame index N+2 of the temporal filtering method. When the encoding step inserts an intraframe at frame index N, the temporal filtering method inserts a complete frame at the corresponding position, frame index N+2, to achieve alignment. Since the temporal filtering method has two invalid frames, the frame corresponding to frame index N+2 is actually the frame at frame index N from the encoding step. Therefore, after the temporal filtering method refreshes the image group by inserting a complete frame, it subsequently accesses a new IBII image group: a complete frame at position N+1, a B-frame at position N+2, a complete frame at position N+3, and a complete frame at position N+4. The temporal filtering method and the frames from the encoding step are aligned again, ensuring that frames at lower temporal layers undergo temporal filtering. Regarding the usage of the offset relationship, this embodiment adopts a conventional frame index mapping configuration. The temporal filtering method internally maintains an independent frame index counter, which increments after processing each frame. Its initial value is set to 0 after counting two invalid frames to ensure the stability of the offset relationship.

[0065] Before the encoding step begins, the temporal filtering method is started. A shared space for at least one frame of image data is reserved in the dynamic random access memory between the two processes. The memory allocation of this shared space is completed by the control logic during the initialization phase, and the allocated size is the storage capacity of a single frame of image data, including storage areas for the luminance and chrominance components. The access relationship of the shared space is as follows: the temporal filtering method acts as the writer, writing the filtered image data into the shared space; the encoding step acts as the reader, retrieving the filtered image from the shared space as the input image for the current frame. The two processes are synchronized through two handshake signals: a completed filtering signal and a filtered image encoding completion signal. The state machine of the handshake timing includes four states: idle state, filtered complete state, encoding working state, and encoding complete state. After the temporal filtering method completes filtering and stores the image in the shared space, it raises the completed filtering signal, then pauses and enters a waiting state. Upon receiving the completed filtering signal, the encoding step lowers the signal and begins working. If it determines that the current frame needs filtering, it retrieves the filtered image from the shared space as the input image for the current frame. After the filtered frame is encoded, the encoding step raises the filtered image encoding completion signal and stops working, waiting for a new signal from the temporal filtering method. Once the temporal filtering method reads the filtered image encoding completion signal, it lowers the signal and begins filtering for the next frame. The raising and lowering of the two signals are executed by their respective flow control logic, and the signal states are stored in the status register area of ​​the dynamic random access memory. The two processes read the signal states via polling or interruption. To address abnormal signal interaction, this embodiment employs a conventional timeout retransmission configuration. If the temporal filtering method does not read the filtered image encoding completion signal within a preset time, it re-raises the completion filtering signal to ensure data transmission integrity.

[0066] In the above process, the IBII image group architecture enables the temporal filtering method to perform filtering at a stable rhythm of once every 4 frames. Frame type determination ensures a clear and unambiguous data path for B-frames to retrieve filtered frames from dynamic random access memory and for I-frames to retrieve original frames from the original image data. The invalid frame statistics and offset alignment mechanism establishes a stable offset relationship between the temporal filtering method and the encoding steps on the frame index. The alignment operation during intra-frame insertion ensures that the frame sequences of the two processes are re-aligned after the image group is refreshed. The shared space dual-signal handshake mechanism achieves stable data transmission between the two processes by alternately raising and lowering the filtering signal and the filtered image encoding completion signal. Therefore, this implementation method enhances compatibility with the standard GOP structure and improves the robustness of the overall process while maintaining the stability of the filtering process.

[0067] Fifth Implementation Method In the fourth embodiment, a stable and efficient workflow collaboration between the temporal filtering method and the encoding step is achieved through the IBII image group architecture, invalid frame offset alignment, and a shared space dual-signal handshake mechanism. The inventors of this invention discovered that the optimization of the filtering intensity in temporal filtering depends on reliable rate-distortion information, and the calculation method of the rate-distortion value directly affects the accuracy of the filtering intensity optimization. If the rate-distortion value is obtained simply by subtracting the predicted pixel from the original pixel, it is difficult to reflect the distortion introduced by the transform and quantization stages, and the selection of the filtering intensity will deviate from the actual distortion situation in the encoding step. The fifth embodiment of this invention further improves the calculation method of the rate-distortion value based on the aforementioned embodiments, introducing a complete reconstruction link of transform-quantization-inverse quantization-inverse transform, so that the rate-distortion value is obtained based on the difference between the original pixel and the reconstructed pixel, and this rate-distortion value is used for the filtering intensity optimization of the temporal filtering method. Figure 5 The diagram illustrates the rate-distortion value calculation process, including the complete data flow from original pixel input, transformation and quantization processing, inverse quantization and inverse transformation processing, reconstructed pixel generation, to rate-distortion value calculation and output.

[0068] Rate-distortion (RDD) calculation is performed by the rate-distortion optimization module, which is located before the filter strength optimization stage of the temporal filtering method and is connected to the filter processing module in the temporal filtering method. The RDD module is triggered before the current coding block begins encoding. After the temporal filtering method obtains the raw pixels of the current frame, the RDD module performs RDD calculation on that coding block. Each coding block is calculated independently once, and the RDD value is output to the filter strength optimization stage after calculation as the basis for determining the filter strength. The start of the RDD module is uniformly scheduled by the control logic of the temporal filtering method. After confirming that the raw pixels of the current frame are ready, the control logic sends a start signal to the RDD module, which then begins executing the transform quantization link.

[0069] The rate-distortion optimization module takes the raw pixels of the current frame as its input parameter. The raw pixels are input as luma and chroma components, which are stored as pixel matrices in the raw image data area of ​​the dynamic random access memory (DRAM). The bit depth of the raw pixels follows the standard bit depth defined in the video coding standard, with the luma and chroma values ​​of each pixel stored as integers. When reading raw pixel data from the DRAM, the rate-distortion optimization module reads block by block according to the pixel matrix size of the coding block. After reading, the raw pixel data is temporarily stored in the input buffer area within the rate-distortion optimization module for subsequent transformation processing.

[0070] The rate-distortion optimization module performs a complete transformation-quantization-inverse quantization-inverse transformation chain on the original pixels. First, a discrete cosine transform or discrete sine transform is performed on the original pixels to map the pixel-domain data to the frequency domain, obtaining transform coefficients. Discrete cosine transform is suitable for most image content with concentrated energy, while discrete sine transform is suitable for image content with strong specificity. The rate-distortion optimization module selects the appropriate transform method based on the characteristics of the image content. After the transformation, quantization processing is performed on the transform coefficients, mapping the frequency domain coefficients to a finite number of quantization levels. The quantization step size is determined by the quantization parameters of the encoding step. Quantization processing introduces quantization distortion, which is the core distortion source that the rate-distortion value needs to reflect. After quantization, inverse quantization processing is performed on the quantized coefficients, mapping the quantization levels back to the frequency domain coefficients. Subsequently, an inverse transform is performed on the inverse-quantized frequency domain coefficients, mapping the frequency domain data back to the pixel domain, obtaining the reconstructed pixel. The reconstructed pixel is the pixel value recovered from the original pixel after the complete transformation, quantization, inverse quantization, and inverse transform chain. The reconstructed pixel differs from the original pixel due to the distortion introduced by the quantization processing. The rate-distortion optimization module subtracts the original pixel from the reconstructed pixel pixel one by one to obtain the difference between the original and reconstructed pixels, and calculates the rate-distortion value based on this difference. This rate-distortion value comprehensively reflects the impact of transform and quantization distortion on image quality and is directly related to the optimization of the filtering intensity of the temporal filtering method. Conventional temporal filtering methods, lacking a transform-quantization module, can only obtain the rate-distortion value by directly subtracting the predicted pixel from the original pixel. This approach cannot reflect the distortion introduced by the transform and quantization stages, making it difficult to accurately match the actual distortion situation in the encoding steps when selecting the filtering intensity. This implementation, through a complete transform-quantization chain, enables the rate-distortion value to comprehensively consider the impact of transform and quantization distortion, providing a more accurate distortion basis for filtering intensity optimization.

[0071] After reconstruction, pixels are temporarily stored in a reconstruction pixel buffer area within the rate-distortion optimization module. This buffer area is allocated by the control logic during the initialization phase, and its storage capacity is the same as that of the original pixel input buffer area, including storage areas for the luminance and chrominance components. During rate-distortion value calculation, both original and reconstructed pixels are simultaneously read from their respective buffer areas, and interpolation is performed pixel-by-pixel. Intermediate results of the interpolation calculation are temporarily stored in the interpolation buffer area within the rate-distortion optimization module. This area has a storage capacity equal to that of a single frame of image data. After the interpolation calculation is complete, the rate-distortion optimization module calculates the rate-distortion value based on the interpolation data.

[0072] After the rate-distortion value is calculated, the rate-distortion optimization module outputs the rate-distortion value to the filter strength optimization stage. The filter strength optimization stage uses the rate-distortion value as the basis for determining the filter strength. The magnitude of the rate-distortion value reflects the degree of distortion in the transform and quantization link of the current coding block. Based on this, the filter strength optimization stage selects a filter strength that matches the degree of distortion and outputs the determined filter strength to the filter processing module. The filter processing module performs temporal filtering processing on the current frame according to this filter strength. The output path of the rate-distortion value is as follows: the output port of the rate-distortion optimization module is connected to the input port of the filter strength optimization stage, and the output port of the filter strength optimization stage is connected to the input port of the filter processing module, forming a complete data link from rate-distortion value calculation to application.

[0073] To address anomalies in the transform quantization link, this embodiment employs a standard default configuration. If data overflow or coefficient anomalies occur during the execution of the discrete cosine transform or discrete sine transform, the rate-distortion optimization module directly uses the original pixels as reconstructed pixels to participate in rate-distortion value calculation, ensuring the integrity of the process. To address anomalies in rate-distortion value calculation, this embodiment employs a standard default configuration. If the difference calculation result exceeds a preset range, the rate-distortion optimization module uses a preset default rate-distortion value as the output, ensuring the normal operation of the filter intensity optimization stage.

[0074] In the above process, the rate-distortion optimization module obtains reconstructed pixels through a complete chain of discrete cosine transform or discrete sine transform, quantization, inverse quantization, and inverse transform. It then calculates the rate-distortion value based on the difference between the original pixels and the reconstructed pixels. This rate-distortion value comprehensively reflects the impact of transform and quantization distortion on image quality and is directly related to the optimization of the filtering intensity of the temporal filtering method. Therefore, this implementation method, while maintaining stable collaboration between the temporal filtering method and the encoding steps, enables the selection of filtering intensity based on more accurate distortion information, thus improving the filtering effect.

[0075] Sixth Implementation Method In the fifth embodiment, the rate-distortion value is calculated based on the difference between the original pixels and the reconstructed pixels, providing a more accurate distortion basis for filter intensity optimization. The inventors of this invention discovered that the motion compensation temporal filtering and encoding steps alternately occupy the coarse motion estimation module, the fine-tuning motion estimation module, and the fractional-level motion estimation module in time. If the timing of the switching of the module input data source is not handled properly, pipeline time slot conflicts will occur, causing delays in the release of the filtering frame and reducing hardware reuse efficiency. The sixth embodiment of this invention further improves the timing of the switching of the module input data source based on the aforementioned embodiments. When the rate-distortion optimization of the encoding step is completed and the next frame is a filtering frame, the input data sources of the coarse motion estimation module, the fine-tuning motion estimation module, and the fractional-level motion estimation module are pre-switched to the motion compensation temporal filtering reference frame. Figure 6The timing diagram of the pre-switching mechanism is shown, including the completion time of rate-distortion optimization in the encoding step, the determination time of the next frame as a filtered frame, the switching time of the input data source of the three motion estimation modules, and the time period occupied by the temporal filtering method in the three modules after the switching.

[0076] The pre-switching action is executed by the control logic of the temporal filtering method. The control logic continuously monitors the running status of the encoding step. When rate-distortion optimization of the encoding step is completed, the encoding step sends a rate-distortion optimization completion signal to the control logic. Upon receiving this completion signal, the control logic further determines whether the next frame is a filtered frame. The method for determining whether the next frame is a filtered frame is connected to the frame type determination in the fourth embodiment. The control logic confirms that the frame type of the next frame is a filtered frame based on the frame type configuration of the current image group. When both conditions—rate-distortion optimization completion and the next frame being a filtered frame—are met simultaneously, the control logic immediately triggers the pre-switching action. The pre-switching action occurs after the rate-distortion optimization of the encoding step is completed and before the temporal filtering method begins to occupy the three motion estimation modules. This timing is located at the time-sharing boundary between the encoding step and the temporal filtering method, reserving sufficient time for the switching of the input data sources of the three motion estimation modules.

[0077] The input data sources for the coarse motion estimation module, the refined optimization motion estimation module, and the fractional motion estimation module include two types: coded reference frames and motion-compensated temporal filtering reference frames. The coded reference frames are the reconstructed frames used for motion estimation in the encoding step, stored in the coded reference frame storage area of ​​the dynamic random access memory (DRAM). This area is allocated by the encoding step during the initialization phase, and its storage capacity is the sum of the capacities of multiple coded reference frames. The motion-compensated temporal filtering reference frames are the reference frames used for filtering in the temporal filtering method, stored in the filtering reference frame storage area of ​​the DRAM. This area is allocated by the control logic of the temporal filtering method during the initialization phase, and its storage capacity is the same as that of the coded reference frame storage area. The coded reference frames and the motion-compensated temporal filtering reference frames are independent in their storage areas. Both have a pixel matrix format consisting of luminance and chrominance components, but the pixel data of the coded reference frames undergoes reconstruction processing in the encoding step, while the pixel data of the motion-compensated temporal filtering reference frames undergoes filtering processing in the temporal filtering method, resulting in differences in their pixel value distributions. The input data sources for the three motion estimation modules are controlled by a data source selection signal, which is a multi-bit control signal that corresponds to the read ports of the encoded reference frame storage area and the filtered reference frame storage area, respectively.

[0078] The semantics of the pre-switching action are as follows: the control logic switches the input data source of the three motion estimation modules from the coded reference frame to the motion-compensated temporal filtering reference frame. The switching action is achieved by updating the data source selection signal; the control logic updates the data source selection signal from the coded value corresponding to the read port of the coded reference frame storage area to the coded value corresponding to the read port of the filtering reference frame storage area. The switching granularity is at the module level; the coarse motion estimation module, the fine-tuning optimized motion estimation module, and the fractional-level motion estimation module perform the switching synchronously, with the control logic simultaneously sending the updated data source selection signal to all three modules. After the switching action is completed, the input data source of all three motion estimation modules points to the motion-compensated temporal filtering reference frame, and the temporal filtering method can then begin to occupy the three modules to perform motion estimation processing. The pre-switching action does not involve any modification to the internal configuration of the motion estimation modules; the motion estimation mode, search range, and other parameters of the three modules remain unchanged, only the input data source changes.

[0079] During the pre-switching operation, both the encoded reference frame storage area and the filtered reference frame storage area remain valid. Data in the encoded reference frame storage area is retained after the pre-switching operation and will not be released or overwritten, ensuring that the encoded reference frame can still be accessed during the encoding of subsequent frames. Data in the filtered reference frame storage area has already been written by the temporal filtering method before the pre-switching operation; the pre-switching operation only changes the data source selection signal and does not involve moving or rewriting data in the filtered reference frame storage area. During the pre-switching operation, the control logic maintains the read / write permissions of both storage areas unchanged; the encoded reference frame storage area maintains read-only permissions, and the filtered reference frame storage area maintains read-only permissions to avoid data access conflicts during the switch. The three motion estimation modules do not execute new motion estimation tasks during the switch; the input buffer areas within the modules remain idle, waiting to receive new input data after the switch is complete.

[0080] After the pre-switching action is completed, the service targets of the coarse motion estimation module, the refined optimization motion estimation module, and the fractional motion estimation module switch from the encoding step to the temporal filtering method. The motion estimation results of the three motion estimation modules are output to the filtering processing module of the temporal filtering method. The filtering processing module performs motion compensation temporal filtering processing based on the motion vectors and matching costs output by the three motion estimation modules. After the pre-switching action is completed, the encoding step no longer occupies the three motion estimation modules. The encoding step waits for the temporal filtering method to complete the filtering processing before re-occupying the three motion estimation modules to perform encoding processing for subsequent frames. Before the switch, the output ports of the three motion estimation modules are connected to the motion compensation module of the encoding step; after the switch, they are connected to the filtering processing module of the temporal filtering method. The switching of the output data path and the switching of the input data source are completed synchronously.

[0081] To address anomalies during the pre-switching process, this embodiment employs the standard default configuration. If, after receiving the rate-distortion optimization completion signal, the control logic determines that the condition for the next frame to be a filtered frame is not met, the control logic does not execute the pre-switching action. The input data source for the three motion estimation modules remains the encoded reference frame, waiting for the encoding step to continue using the three motion estimation modules for subsequent processing. To address anomalies during the pre-switching execution, this embodiment employs the standard default configuration. If the control logic does not receive confirmation signals from the three motion estimation modules after sending the data source selection signal, the control logic resends the data source selection signal until all three motion estimation modules confirm the switching is complete, ensuring the integrity of the process.

[0082] In the above process, when the rate-distortion optimization of the encoding step is completed and the next frame is a filtering frame, the control logic pre-switches the input data source of the three motion estimation modules to the motion compensation temporal filtering reference frame. This ensures that the three motion estimation modules have completed the data source switching before the temporal filtering method starts to occupy the frame, eliminating pipeline bubbles caused by the input data source switching between the encoding step and the temporal filtering method, avoiding time slot conflicts when the filtering frame is waiting for the module to release, and improving hardware reuse efficiency.

[0083] Seventh Implementation Method In the sixth embodiment, a pre-switching mechanism eliminates time slot conflicts between the motion estimation module and the temporal filtering method in the encoding step, improving hardware reuse efficiency. The inventors of this invention discovered that the filtering effect of bilateral filtering directly depends on the method of determining the filtering parameters. If the filtering parameters are fixed or obtained through offline table lookup, the adaptability of the filtering intensity to the image content will be limited, making it difficult to dynamically adjust the filtering intensity according to the rate-distortion state of the current frame. The seventh embodiment of this invention further improves the calculation method of the bilateral filtering parameters based on the aforementioned embodiments. It introduces the rate-distortion value calculated by the rate-distortion optimization module and the quantization parameters of the encoding step as input for parameter calculation. Spatial domain filtering parameters and tolerance domain filtering parameters are calculated analytically using closed-form functions, and further employs mean-variance statistics and 16-bit fixed-point arithmetic to achieve hardware-friendly parameter mapping. Figure 7 The diagram illustrates the analytical mapping process of bilateral filter parameters, including the acquisition paths for rate-distortion values ​​and quantization parameters, the mean and variance statistical window, the closed-loop function calculation unit, the 16-bit fixed-point multiplication and division unit, the square root lookup table, and the 10-bit fixed-point output path.

[0084] The parameter calculation unit obtains the rate-distortion value of the current encoded frame from the rate-distortion optimization module, and simultaneously obtains the quantization parameters from the quantization parameter register of the encoding step. The rate-distortion value is the cost value calculated by the rate-distortion optimization module in the encoding step, and the quantization parameters are the parameters controlling the quantization step size in the encoding step. Both are updated once after each encoding unit in the encoding step is processed. The parameter calculation unit reads the above two values ​​before the temporal filtering method begins. The timing of the reading coincides with the triggering timing of the pre-switching action in the sixth embodiment, ensuring that the rate-distortion value and quantization parameters obtained by the parameter calculation unit are the latest values ​​of the current frame.

[0085] The rate-distortion value is formatted as a 32-bit unsigned integer, and the quantization parameter is formatted as an 8-bit unsigned integer. The rate-distortion value is stored in the internal register of the rate-distortion optimization module, and the quantization parameter is stored in the quantization parameter register of the encoding step. The parameter calculation unit reads the values ​​from both registers via the internal data bus. The numerical ranges of both the rate-distortion value and the quantization parameter cover the typical working range of the encoding step. The parameter calculation unit performs a range check on the read values. If either the rate-distortion value or the quantization parameter exceeds the preset range, the parameter calculation unit uses a preset default value to ensure the stability of subsequent calculations.

[0086] The parameter calculation unit calculates the bilateral filter parameters based on the rate-distortion value and quantization parameter using a closed-form function. The input to the closed-form function is the rate-distortion value and the quantization parameter, and the output is the spatial domain filter parameter and the tolerance domain filter parameter. The semantics of the closed-form function are: the parameter calculation unit performs deterministic analytical calculations on the input values, directly obtaining the output values, without relying on table lookups or using fixed preset values. The spatial domain filter parameter and the tolerance domain filter parameter share the same set of input values, but their analytical calculation paths are independent. The parameter calculation unit executes each of the two calculation paths and outputs the two calculation results in parallel.

[0087] Spatial domain filtering parameters control the spatial expansion of the bilateral filtering window. Larger values ​​result in a wider pixel range covered by the filtering window and a greater weighting of spatially neighboring pixels. Tolerance domain filtering parameters control the tolerance of the bilateral filter to pixel value differences. Larger values ​​result in greater tolerance for pixel value differences and better preservation of pixels in edge regions. The spatial and tolerance domain filtering parameters together determine the filtering strength of the bilateral filter; their combined values ​​directly reflect the rate-distortion state of the current frame and the quantization step size's requirement for filtering strength.

[0088] The parameter calculation unit outputs the calculated spatial domain filtering parameters and tolerance domain filtering parameters to the filtering processing module, which then performs bilateral filtering based on these parameters. Simultaneously with outputting the parameters, the parameter calculation unit sends a parameter update flag to the filtering processing module. Upon receiving the flag, the filtering processing module updates its internally stored filtering parameters to the newly calculated values. The parameter calculation unit performs parameter calculations once before the start of the temporal filtering method for each coded frame, ensuring that the filtering parameters used by the filtering processing module always match the rate-distortion state of the current frame.

[0089] To address anomalies during closed-form function calculation, this embodiment employs a standard default configuration. If the rate-distortion value or quantization parameter read by the parameter calculation unit exceeds a preset range, the parameter calculation unit uses preset default parameter values ​​as the output of the bilateral filter parameters to ensure process integrity. Regarding parameter transmission anomalies between the parameter calculation unit and the filtering module, this embodiment also uses a standard default configuration. If the filtering module does not receive a parameter update flag within a preset time, the filtering module uses the filtering parameters from the previous frame to perform filtering processing, ensuring process integrity.

[0090] Furthermore, before the parameter calculation unit calculates the bilateral filter parameters based on the rate-distortion values, it can first perform statistical analysis on the mean and variance of the rate-distortion values. The statistical window uses a sliding window approach, with a window length covering a preset number of coding units within the current frame. The parameter calculation unit maintains a sliding window buffer, storing the rate-distortion values ​​of the most recent preset number of coding units. The rate-distortion value of each coding unit is written to the sliding window buffer after the encoding step is completed, while the oldest rate-distortion value in the buffer is removed, and the set of rate-distortion values ​​within the window is dynamically updated.

[0091] The parameter calculation unit calculates the mean and variance of the ratedistortion values ​​within the sliding window buffer. The mean is the arithmetic mean of all ratedistortion values ​​within the window, and the variance is the average of the squares of the differences between each ratedistortion value and the mean. When calculating the mean, the parameter calculation unit sums all ratedistortion values ​​within the window and divides by the window length; when calculating the variance, it first calculates the difference between each ratedistortion value and the mean, squares these differences, sums them, and then divides by the window length. The results of the mean and variance calculations are stored in the internal registers of the parameter calculation unit in fixed-point number format.

[0092] The statistical methods of using the mean and variance have a suppressive effect on outliers in the rate-distortion (RDC) value sequence. When the RDC value of a certain coding unit abnormally increases or decreases due to abrupt changes in image content, this outlier only has a limited impact on the mean and variance within the sliding window. The statistical contributions of other RDC values ​​within the window can dilute the interference of the outlier. When the parameter calculation unit calculates the bilateral filter parameters based on the mean and variance, the influence of outliers is limited to a proportional range corresponding to the window length, avoiding drastic fluctuations in filter parameters caused by abnormal RDC values ​​of a single coding unit.

[0093] The parameter calculation unit uses the statistically obtained mean and variance as input to a closed-form function to replace or correct the original rate-distortion values. The input to the closed-form function is the mean and variance of the rate-distortion values, along with quantization parameters; the output is spatial domain filtering parameters and tolerance domain filtering parameters. The mean reflects the central tendency of the current frame rate-distortion state, while the variance reflects the dispersion of the current frame rate-distortion state. The combination of these two values ​​provides the closed-form function with richer statistical information than a single rate-distortion value, making the calculated filtering parameters more adaptable to the image content.

[0094] The parameter calculation unit outputs the spatial domain filtering parameters and tolerance domain filtering parameters, calculated based on the mean and variance, to the filtering processing module. The output path is consistent with the aforementioned parameter calculation path. Simultaneously with outputting the parameters, the parameter calculation unit sends a parameter update flag to the filtering processing module, which then performs bilateral filtering based on the updated parameters. The statistical results of the mean and variance are simultaneously stored in the internal registers of the parameter calculation unit for use in subsequent encoded frame parameter calculations.

[0095] To address anomalies in the mean and variance calculation process, this embodiment employs a standard default configuration. If the number of rate-distortion values ​​in the sliding window buffer is insufficient for the window length, the parameter calculation unit calculates the mean and variance based on the existing rate-distortion values. Once the window is full, it switches to the full window statistics mode to ensure process integrity. To address anomalies such as overflow in the mean or variance calculation results, this embodiment uses a standard default configuration. The parameter calculation unit performs saturation truncation on the calculation results, cutting off values ​​exceeding the preset range to boundary values ​​to ensure process integrity.

[0096] It is worth mentioning that the parameter calculation unit uses 16-bit fixed-point arithmetic when performing multiplication and division operations on rate-distortion values ​​and quantization parameters. The semantics of 16-bit fixed-point arithmetic are: the values ​​involved in the operation are represented in 16-bit fixed-point format, the decimal point position is preset by the parameter calculation unit based on the numerical range before the operation, and the decimal point position remains unchanged during the operation. 16-bit fixed-point arithmetic avoids the hardware overhead of floating-point arithmetic; the parameter calculation unit does not need to be configured with a floating-point arithmetic unit to complete multiplication and division operations, reducing hardware implementation costs.

[0097] When the parameter calculation unit performs multiplication on the rate-distortion value and the quantization parameter, it inputs two 16-bit fixed-point numbers into the multiplier. The multiplier outputs a 32-bit fixed-point product. The parameter calculation unit truncates the product, retaining the high 16 bits as the multiplication result. When the parameter calculation unit performs division, it converts both the dividend and divisor to 16-bit fixed-point numbers. The divider performs fixed-point division and outputs a 16-bit fixed-point quotient. Intermediate results of multiplication and division operations are stored in 16-bit fixed-point format. The parameter calculation unit does not introduce floating-point intermediate values ​​during the calculation process.

[0098] When the parameter calculation unit needs to obtain the square root result during the calculation process, it queries a pre-stored square root lookup table. The square root lookup table is stored in the read-only memory (ROM) inside the parameter calculation unit. The index of the lookup table is a 16-bit fixed-point input value, and the output of the lookup table is the square root result of the corresponding input value. The parameter calculation unit inputs the value to be taken as the index into the lookup table, and the lookup table outputs the corresponding square root result with a precision of 16-bit fixed-point numbers. The storage capacity of the square root lookup table covers the typical range of input values, and the lookup table is loaded into the ROM during the parameter calculation unit's initialization phase.

[0099] The parameter calculation unit outputs the calculated spatial domain filtering parameters and tolerance domain filtering parameters in a 10-bit fixed-point number format. The numerical range of the 10-bit fixed-point number is 0 to 1023. Before outputting, the parameter calculation unit performs quantization processing on the calculation result, converting the 16-bit fixed-point number result into a 10-bit fixed-point number. The quantization processing uses a rounding method; the parameter calculation unit performs rounding judgment on the lower 6 bits of the 16-bit fixed-point number, and the rounded result is superimposed on the higher 10 bits before output. The 10-bit fixed-point number output format matches the bit width of the input interface of the filtering module, so the filtering module can directly use the filtering parameters without performing additional bit width conversion.

[0100] 16-bit fixed-point multiplication and division operations, along with square root lookup table lookup operations, allow the parameter calculation unit to perform analytical calculations of closed functions without the need for a floating-point arithmetic unit. The hardware implementation area of ​​fixed-point operations is smaller than that of a floating-point arithmetic unit, and the lookup latency of the lookup table is less than that of floating-point square root operations, thus reducing the overall hardware overhead and computational latency of the parameter calculation unit. The 10-bit fixed-point output format matches the input interface width of the filtering module, avoiding the overhead of bit-width conversion during parameter transmission.

[0101] To address anomalies during fixed-point arithmetic, this embodiment employs a standard default configuration. If the result of a multiplication or division operation exceeds the representation range of a 16-bit fixed-point number, the parameter calculation unit performs saturation truncation on the result, truncating it to the maximum or minimum value of the 16-bit fixed-point number to ensure process integrity. Similarly, to address anomalies where the index of the square root lookup table exceeds the table's coverage, this embodiment uses a standard default configuration. The parameter calculation unit uses the lookup table boundary value as the square root result to ensure process integrity.

[0102] In the above process, the parameter calculation unit obtains the rate-distortion value calculated by the rate-distortion optimization module and the quantization parameters of the encoding steps. Based on the rate-distortion value and the quantization parameters, it calculates the spatial domain filtering parameters and the tolerance domain filtering parameters through closed functions. Furthermore, it statistically analyzes the mean and variance of the rate-distortion value to suppress the influence of outliers. At the same time, it uses 16-bit fixed-point multiplication and division operations and square root lookup tables to achieve hardware-friendly analytical calculation. Finally, it outputs the bilateral filtering parameters as 10-bit fixed-point numbers, so that the filtering parameters are adaptively determined according to the rate-distortion state of the current frame, which improves the filtering effect and content adaptability, while ensuring the feasibility of parameter calculation on the hardware platform.

[0103] Eighth Implementation Method In the seventh embodiment, the bilateral filtering parameters are calculated analytically using closed-form functions, allowing the filtering intensity to adapt to the rate distortion state, thus improving the filtering effect and content adaptability. The inventors of this invention discovered that since the filtered frame and the original frame belong to different data sources, the encoding step needs to distinguish the frame type and manage the data source during data retrieval. If an explicit frame type determination unit is used in conjunction with a flag register and selection logic, the encoding step needs to query the frame type flag before each data retrieval and then select the corresponding data source based on the flag value. This process requires additional control logic and area overhead, and increases the latency of the data retrieval path. The eighth embodiment of this invention, based on the aforementioned embodiments, proposes to physically encode the frame type information in the address space of a dynamic random access memory. The data retrieval source for the encoding step is automatically routed by the high-order bits of the address, eliminating the need for additional data retrieval control logic and reducing control complexity and area overhead. Figure 8 The diagram illustrates the address routing of a dual-memory area in a dynamic random access memory. The left side of the diagram shows the address space division between the first and second memory areas, while the right side shows the routing path for the high-order bits of the address. The high-order bits of the address automatically point to the corresponding memory area based on the lower two bits of the current frame index.

[0104] The Dynamic Random Access Memory (DRAM) is divided into a first storage area and a second storage area. The first storage area stores the original frames, and the second storage area stores the filtered frames. Both the first and second storage areas have a capacity of 256 megabytes, and their address spaces each occupy one-quarter of the total DRAM address space. The DRAM uses a 32-bit address bus, with address bits [31:30] used as storage area selection bits and address bits [29:0] used as internal offset addresses within the storage areas. The address range of the first storage area is 0x00000000 to 0x0FFFFFFF, and the address range of the second storage area is 0x10000000 to 0x1FFFFFFF. When address bits [31:30] are 00, they point to the first storage area; when they are 01, they point to the second storage area; and when they are 10 or 11, they correspond to the reserved address space. The reserved address space is used to store intermediate data from the encoding steps, motion vector information, and bitstream buffer data, and does not participate in the data routing for frame data retrieval.

[0105] In the temporal filtering method, after completing the filtering process, the filtering module writes the filtered frame to the second storage area. The filtering module sends the write address to the dynamic random access memory via the write address bus. The address bits [31:30] of the write address are fixedly set to 01, and the address bits [29:0] are generated based on the pixel positions of the filtered frame. When writing the filtered frame, the filtering module does not perform any frame type judgment logic; it directly writes the filtered frame to the second storage area using the fixed encoding of the address bits [31:30]. The original frame is written to the first storage area by the front-end acquisition module before the encoding step begins. The address bits [31:30] of the write address are fixedly set to 00, and the address bits [29:0] are generated based on the pixel positions of the original frame.

[0106] Both the first and second storage areas have a capacity of 256 megabytes, sufficient to hold complete frame data at the preset resolution. For frame data with a preset resolution of 1920x1080, the luminance and chrominance components of each frame are approximately 3 megabytes in size. The 256-megabyte storage capacity can accommodate multiple frames, providing ample address space for data retrieval operations during the encoding process. An additional 512 megabytes of reserved address space is used to store intermediate data from the encoding process, preventing address conflicts between frame data storage and other data storage.

[0107] In the seventh embodiment described above, the spatial domain filtering parameters and tolerance domain filtering parameters calculated by the parameter calculation unit are output to the filtering processing module. The filtering processing module then performs bilateral filtering based on these two parameters. After completing the filtering process, the filtering processing module writes the filtered frame into the second storage area. The address space layout of the second storage area is independent of the first storage area, and the filtered frame and the original frame are separated in physical address space, providing an address basis for automatic routing of data sources in the encoding step.

[0108] Frame type information is no longer stored in a separate flag register, but is physically encoded in address bits [31:30] of the dynamic random access memory. When the frame type is a filtered frame, address bits [31:30] are encoded as 01; when the frame type is a raw frame, address bits [31:30] are encoded as 00. During the data retrieval request, address bits [31:30] are automatically generated by combinational logic based on the current frame index, without needing to query any frame type flag register. The physical encoding method of frame type information allows the frame type judgment function to be moved from the explicit logic judgment unit to the combinational logic of the address decoder. The frame type judgment function still exists, but its physical implementation location has been transferred from the encoding process state machine to the dynamic random access memory address decoder.

[0109] The address bits [31:30] of the 32-bit address bus serve as memory selection bits, and the address bits [29:0] serve as internal offset addresses within the memory area. The internal offset addresses are used to locate pixel positions within a frame, including row addresses, column addresses, and component addresses. The address bits [29:0] have a width of 30 bits, capable of covering a memory area capacity of 256 megabytes. During the encoding process, when generating the data retrieval address, address bits [29:0] are generated based on the pixel position of the current encoding unit, and address bits [31:30] are automatically generated based on the lower two bits of the current frame index. These two bits are then concatenated to form a complete 32-bit data retrieval address.

[0110] After the frame type determination function is transferred from the explicit logic determination unit to the combinational logic of the address decoder, the data retrieval process in the encoding step no longer includes the frame type determination step. The encoding step only needs to generate the data retrieval address based on the current frame index, and the address decoder automatically routes the data retrieval request to the corresponding memory area based on the address bits [31:30]. The physical transfer of the frame type determination logic simplifies the state machine of the encoding step. The encoding step does not need to maintain the read / write timing of the frame type flag register, nor does it need to perform the query and comparison operations of the frame type flag.

[0111] In conventional schemes, the frame type determination unit requires a comparator, counter, flag register, and lookup table logic, with a total area of ​​approximately 120 gates and a total latency of approximately 2.5 clock cycles for the data retrieval path. In this embodiment, after the frame type information is physically encoded in the address space, the address decoder only needs 3 combinational logic gates to complete the memory area selection, and the total latency of the data retrieval path is approximately 0.5 clock cycles. The physical transfer of the frame type determination logic shortens the data retrieval path in the encoding step, reduces the data retrieval latency, and simultaneously reduces the area overhead of the flag register and lookup table logic.

[0112] It is worth mentioning that when the encoding step initiates a data retrieval request, the high-order bits of the address are automatically generated by the combinational logic based on the low 2 bits of the current frame index. The current frame index is a 16-bit counter, and the count value range covers the frame sequence length of the encoding step. The input of the combinational logic is the low 2 bits of the current frame index, and the output is the storage area selection value of address bits [31:30]. When the low 2 bits of the current frame index are 01, address bits [31:30] output 01, pointing to the second storage area; when the low 2 bits of the current frame index are 00, 10, or 11, address bits [31:30] output 00, pointing to the first storage area. The Boolean expression of the combinational logic is: the first bit of the storage area selection bit is always 0, and the second bit of the storage area selection bit is equal to the inverse AND of the first and second bits of the low 2 bits of the current frame index, that is, the storage area selection value 01 is output only when the low 2 bits are 01. This combinational logic requires only one 2-input AND gate and one inverter, totaling approximately 3 gates. It has no status register, no lookup table operation, and no multiplexer.

[0113] During the encoding process, the data retrieval address is generated based on the lower two bits of the current frame index. When the lower two bits of the current frame index are 0 or 1, the address bits [31:30] of the data retrieval address are 0 or 1, the address decoder routes the data retrieval request to the second storage area, and the encoding process reads the filtered frame data from the second storage area. When the lower two bits of the current frame index are 00, 10, or 11, the address bits [31:30] of the data retrieval address are 00, the address decoder routes the data retrieval request to the first storage area, and the encoding process reads the original frame data from the first storage area. The data retrieval routing process is entirely completed by the combinational logic of the address decoder; the encoding process does not need to perform frame type judgment operations or configure additional data retrieval control logic.

[0114] The mapping relationship between the lower 2 bits of the current frame index and the data source is as follows: when the lower 2 bits are 0 or 1, the data source is the filtered frame in the second storage area; when the lower 2 bits are 00, 10, or 11, the data source is the original frame in the first storage area. This mapping relationship is consistent with the frame type configuration of the encoding step. When the frame index modulo 4 is 1, the encoding step encodes the filtered frame; when the frame index modulo 4 is 0, 2, or 3, the encoding step encodes the original frame. The data source routing mapping relationship is determined by the physical encoding of the address bits [31:30]. The encoding step does not need to maintain a mapping table between frame type and data source.

[0115] During motion estimation in the encoding step, the motion estimation module reads reference frame data from the dynamic random access memory. The data retrieval address of the motion estimation module is also automatically routed using the high-order bits of the address. When the reference frame type is a filtered frame, address bits [31:30] of the data retrieval address are 01, pointing to the second storage area; when the reference frame type is a raw frame, address bits [31:30] of the data retrieval address are 00, pointing to the first storage area. The data retrieval route of the motion estimation module shares the same address decoder as the data retrieval route of the encoding step, requiring no additional data retrieval control logic.

[0116] After completing the filtering process, the filtering module writes the filtered frame to the second storage area. The address bits [31:30] of the write address of the filtering module are fixed at 01, while the address bits [29:0] are generated based on the pixel positions of the filtered frame. The write address generation process of the filtering module does not depend on the current frame index; the address bits [31:30] of the write address are always kept at 01 to ensure that the filtered frame is written to the second storage area. When the encoding step reads the filtered frame from the second storage area, the read filtered frame data is the latest data written by the filtering module.

[0117] Furthermore, the main body responsible for automatic high-order address routing is the dynamic random access memory (DRAM) address decoder. Located inside the DRAM controller, the address decoder is responsible for decoding the access addresses sent by the encoding step, motion estimation module, and filtering module into row and column selection signals for the memory array. The address decoder receives the 32-bit fetch address sent by the encoding step, parses the memory area selection value of address bits [31:30], and routes the fetch request to either the first or second memory area based on the memory area selection value. The combinational logic of the address decoder is automatically triggered when the encoding step initiates a fetch request, requiring no additional control signals.

[0118] Automatic high-order address routing is triggered when a data retrieval request is initiated during the encoding step. The encoding step initiates multiple data retrieval requests during the processing of each encoding unit, each request carrying a 32-bit data retrieval address. Upon receiving the data retrieval address, the address decoder immediately parses the storage area selection value of address bits [31:30] and routes the data retrieval request to the corresponding storage area. The triggering timing of the data retrieval request is synchronized with the encoding rhythm of the encoding step. When encoding each encoding unit of the current frame, the encoding step generates a data retrieval address based on the lower two bits of the current frame index, and the address decoder automatically routes the data based on the data retrieval address.

[0119] The state machine in the encoding step maintains a counter value for the current frame index while encoding the current frame. The current frame index is updated at the beginning of each frame encoding, and the updated value is used to generate the address bits [31:30] of the data retrieval address. The state machine in the encoding step does not need to maintain the read / write timing of the frame type flag register, nor does it need to perform frame type judgment operations. The data retrieval process of the state machine is simplified to three steps: generating the data retrieval address, initiating the data retrieval request, and receiving the data retrieval data, thus reducing the control complexity of the data retrieval process.

[0120] During the GOP refresh, the current frame index is reset to 0. After the current frame index is reset, the lower two bits are 00, and the address bits [31:30] automatically point to the first storage area. The first frame after the GOP refresh is a complete frame of the image group header. Encoding directly reads the original frame data from the first storage area, consistent with the frame type configuration of the encoding step. The triggering of the GOP refresh signal does not affect the combinational logic of automatic routing of the high-order address bits. The combinational logic only generates the storage area selection value based on the lower two bits of the current frame index. No additional judgment logic is required for the access of the GOP refresh signal.

[0121] In this embodiment, the address bus width of the dynamic random access memory (DRAM) is 32 bits, with address bits [31:30] serving as memory region selection bits and address bits [29:0] serving as internal offset addresses within the memory region. The address bus width matches the address space capacity of the DRAM, and the 32-bit address bus can address a 4-gigabyte address space. The address spaces of the first and second memory regions each occupy 256 megabytes, and the 30-bit offset address of address bits [29:0] can cover the 256-megabyte memory region capacity.

[0122] Both the first and second storage areas are set to a capacity of 256 megabytes, sufficient to hold complete frame data at the preset resolution. For frame data with a preset resolution of 1920x1080, the luminance and chrominance components of each frame are approximately 3 megabytes in size, and the 256-megabyte storage capacity can accommodate multiple frames. The storage capacity configuration is completed during the dynamic random access memory controller initialization phase, where the controller configures the storage area boundary addresses according to the preset resolution.

[0123] The frame data has a bit depth of 8 bits, and the luminance and chrominance components of each pixel are represented by 8-bit unsigned integers. The frame data is stored in dynamic random access memory (DRAM) with luminance and chrominance components stored separately in different planes. The pixel data for each component is stored contiguously in an offset address space within the memory area. When the encoding step reads frame data from DRAM, an offset address is generated based on the pixel position of the encoding unit, and the read frame data is output to the pixel buffer of the encoding step as an 8-bit unsigned integer.

[0124] The current frame index is a 16-bit counter, and its count value covers the length of the frame sequence in the encoding steps. The lower 2 bits of the current frame index are used to generate the memory selection value for address bits [31:30], and the higher 14 bits are used to identify the frame position in the frame sequence. The count value of the current frame index is updated at the beginning of each frame encoding, and the updated value is used to generate both the address bits [31:30] and the higher bits of address bits [29:0] for the data retrieval address.

[0125] Furthermore, the mapping relationship between address bits [31:30] and the lower 2 bits of the current frame index is as follows: when the lower 2 bits of the current frame index are 01, address bits [31:30] are 01; when the lower 2 bits of the current frame index are 00, 10, or 11, address bits [31:30] are 00. This mapping relationship is implemented by the combinational logic of the address decoder. The input of the combinational logic is the lower 2 bits of the current frame index, and the output is the memory selection value of address bits [31:30]. The Boolean expression of the combinational logic is: the first bit of the memory selection bit is always 0, and the second bit of the memory selection bit is equal to the inverse AND of the first and second bits of the lower 2 bits of the current frame index, that is, the memory selection value 01 is output only when the lower 2 bits are 01.

[0126] The logic for determining the lower two bits of the current frame index is implemented using combinational logic in the address decoder. The combinational logic checks if the lower two bits of the current frame index are 0 or 1. When they are, the combinational logic outputs a memory selection value of 01; when they are not, it outputs a memory selection value of 00. This lower two-bit determination logic requires only one 2-input AND gate and one inverter, approximately three gates in total. It has no status register, no lookup table operation, and no multiplexer. The propagation delay of the lower two-bit determination logic is approximately 0.5 clock cycles, and the total delay of the data fetching path is approximately 0.5 clock cycles.

[0127] Upon receiving the data fetch address, the combinational logic of the address decoder parses the memory selection value of address bits [31:30]. When the memory selection value is 01, the address decoder routes the data fetch request to the second memory area; when the memory selection value is 00, the address decoder routes the data fetch request to the first memory area. Simultaneously, the combinational logic of the address decoder parses the internal offset address of address bits [29:0] and generates row and column selection signals for the memory array based on the offset address. During the process of parsing the data fetch address, the combinational logic of the address decoder does not perform frame type determination or query the frame type flag register.

[0128] The hardware implementation of the address bit mapping relationship requires only 3 combinational logic gates, with no status registers, no lookup table operations, and no multiplexers. The input of the combinational logic is the lower 2 bits of the current frame index, and the output is the memory selection value of address bits [31:30]. The propagation delay of the combinational logic is approximately 0.5 clock cycles, and the total latency of the data fetching path is approximately 0.5 clock cycles. Compared with the conventional solution, which requires a frame type determination unit, a flag register, lookup table logic, and a multiplexer, the conventional solution has a total area of ​​approximately 120 gates and a total latency of approximately 2.5 clock cycles for the data fetching path. The combinational logic area of ​​this implementation is approximately 3 gates, and the total latency of the data fetching path is approximately 0.5 clock cycles, significantly reducing both area overhead and data fetching latency.

[0129] In some implementations, the dynamic random access memory (DRAM) is divided into a first storage area and a second storage area. The first storage area stores raw frames, and the second storage area stores filtered frames. The address range of the first storage area is 0x00000000 to 0x0FFFFFFF, and the address range of the second storage area is 0x10000000 to 0x1FFFFFFF. Both the first and second storage areas have a capacity of 256 megabytes, and their address spaces are independent, with frame data physically separated. The division of the dual storage areas is completed during the DRAM controller initialization phase, where the controller configures the storage area boundary addresses according to a preset resolution.

[0130] The address mapping relationship of frame data in dynamic random access memory is as follows: the original frame is stored in the first storage area, and the address bits [31:30] are 00; the filtered frame is stored in the second storage area, and the address bits [31:30] are 01. The pixel position within the frame is mapped to the internal offset address of the storage area at address bits [29:0]. The offset address is generated based on the row coordinate, column coordinate, and component identifier of the pixel. When the encoding step initiates a data retrieval request, the address bits [31:30] are generated based on the lower 2 bits of the current frame index, and the address bits [29:0] are generated based on the pixel position of the current encoding unit. These are then concatenated to form a complete 32-bit data retrieval address.

[0131] The address space in the Dynamic Random Access Memory (DRAM) with addresses [31:30] of 10 or 11 is a reserved address space with a capacity of 512 megabytes. This reserved address space is used to store intermediate data for the encoding steps, motion vector information, and bitstream buffer data. Access to the reserved address space is managed by the DRAM controller, which configures access permissions during the initialization phase. Data retrieval requests from the encoding steps and the motion estimation module are not routed to the reserved address space because the storage area selection value for addresses [31:30] can only be 00 or 01.

[0132] The boundary addresses between the first and second memory areas are aligned with the memory array structure of the dynamic random access memory. The boundary address of the first memory area, 0x10000000, is aligned with the boundary of the memory bank of the memory array, and the boundary address of the second memory area, 0x20000000, is aligned with the boundary of the memory bank of the memory array. This boundary alignment enables the combinational logic of the address decoder to efficiently route data fetch requests to the corresponding memory banks, reducing address decoding latency.

[0133] It is worth mentioning that when the encoding step reads frame data from the dynamic random access memory, the address decoder routes the data retrieval request to the corresponding storage area based on the address bits [31:30] of the data retrieval address. When the storage area selection value is 01, the data retrieval request is routed to the second storage area, which outputs filtered frame data, which is then sent to the pixel buffer of the encoding step via the data bus. When the storage area selection value is 00, the data retrieval request is routed to the first storage area, which outputs raw frame data, which is then sent to the pixel buffer of the encoding step via the data bus. The data retrieval source is automatically routed by the high-order bits of the address, so the encoding step does not need to perform frame type judgment operations or configure additional data retrieval control logic.

[0134] The dynamic random access memory (DRAM) has a 64-bit data bus width, transmitting 64 bits of data per clock cycle. After initiating a data fetch request, the encoding step waits a preset number of clock cycles before receiving the fetched data. The timing of the data transmission is managed by the DRAM controller, which schedules the read and write operations of the data bus based on the memory area selection value of the data fetch request. Upon receiving the fetched data, the encoding step writes the data to a pixel buffer, which is then used for subsequent processing in the encoding step.

[0135] The pixel buffer in the encoding step stores frame data read from dynamic random access memory. The capacity of the pixel buffer matches the number of pixels in the encoding unit, and it can store the complete pixel data of one encoding unit. When reading frame data, the encoding step writes the fetched data into the pixel buffer, and the data in the pixel buffer is used for prediction, transformation, and quantization processing in the encoding step. After the data source is automatically routed by the high-order bits of the address, the data source of the pixel buffer is determined by the address bits [31:30] of the data fetch address, and the encoding step does not need to maintain the mapping relationship between frame type and data source.

[0136] During motion estimation in the encoding step, the motion estimation module reads reference frame data from the dynamic random access memory. The data retrieval address of the motion estimation module is also automatically routed using the high-order bits of the address. When the reference frame type is a filtered frame, address bits [31:30] of the data retrieval address are 01, pointing to the second storage area; when the reference frame type is a raw frame, address bits [31:30] of the data retrieval address are 00, pointing to the first storage area. The data retrieval route of the motion estimation module shares the same address decoder as the data retrieval route of the encoding step, requiring no additional data retrieval control logic.

[0137] In response to the abnormal situation where the address bits [31:30] of the data retrieval address in the encoding step or motion estimation module are 10 or 11, this embodiment adopts the conventional default configuration. The address decoder routes the data retrieval request to the reserved address space. Access to the reserved address space is managed by the dynamic random access memory controller. The controller decides whether to respond to the data retrieval request based on the preset access permissions to ensure the integrity of the process.

[0138] To address the read-write conflict that occurs simultaneously when the encoding step reads the filtered frame from the second storage area and the filtering processing module writes the filtered frame to the second storage area, this embodiment adopts a conventional default configuration. The dynamic random access memory controller schedules read and write operations according to the priority of read and write requests, with write requests taking precedence over read requests, in order to ensure the integrity of the filtered frame data and the integrity of the process.

[0139] In response to the abnormal situation where the address bits [29:0] of the data retrieval address exceed the storage area capacity, this embodiment adopts the conventional default configuration. The address decoder routes the data retrieval request to the boundary address of the storage area, and the encoding step reads the frame data at the boundary address to ensure the integrity of the process.

[0140] In response to the abnormal situation where the current frame index count value is abnormal, this embodiment adopts the conventional default configuration. When the lower 2 bits of the current frame index are 0 or 1, the address bits [31:30] point to the second storage area; when the lower 2 bits of the current frame index are not 0 or 1, the address bits [31:30] point to the first storage area. The mapping relationship of the combinational logic is not affected by the abnormal frame index, so as to ensure the integrity of the process.

[0141] In the above process, the dynamic random access memory is divided into a first storage area and a second storage area. The first storage area stores the original frame, and the second storage area stores the filtered frame. The frame type information is physically encoded in the address bits of the dynamic random access memory [31:30]. The data source for the encoding step is automatically routed by the high bits of the address. When the lower 2 bits of the current frame index are 0 or 1, the high bits of the address point to the second storage area; otherwise, they point to the first storage area. The data routing process is automatically completed by the combinational logic of the address decoder, without the need for additional data control logic, which reduces the control complexity and area overhead, while shortening the delay of the data path and improving the data acquisition efficiency of the encoding step.

[0142] Ninth Implementation Method In the eighth embodiment, frame type information is physically encoded in the address space of dynamic random access memory (DRAM). The data source for the encoding step is automatically routed by the high-order address bits. The filtered frame is implicitly ready when written to the second storage area, eliminating the need for additional data retrieval control logic and reducing control complexity and area overhead. The inventors of this invention discovered that the temporal filtering method and the encoding step share the DRAM's write and read operations, resulting in asynchronous rhythms. If a ping-pong dual-signal alternating pause is used, the frame period is the sum of the filtering and encoding durations. Under a 60fps frame period constraint, timing convergence is difficult, and the alternating pauses reduce hardware utilization. The ninth embodiment of this invention, based on the aforementioned embodiments, proposes introducing bidirectional flow control using credit tokens: the temporal filtering method holds N credit tokens, consuming one credit token for each frame filtered, and the encoding step returns one credit token for each frame consumed. Token counting achieves pause-free pipeline synchronization, avoiding buffer overflows and data conflicts, and bidirectional pauses require zero additional handshake signals. Figure 9 The diagram illustrates the timing of bidirectional flow control using credit tokens. The horizontal axis represents time, and the vertical axis represents the credit token count. The top axis shows the consumption events of the temporal filtering method and the return events of the encoding step, while the bottom axis shows the curve of the credit token count. The curve triggers a pause in temporal filtering when the count is 0 and a pause in the encoding step when the count is N.

[0143] The credit token mechanism is a credit-based flow control method. Its core idea is to synchronize the temporal filtering method and the encoding step through the consumption and return of credit tokens. The temporal filtering method holds N credit tokens, consuming one credit token for each frame filtered. The encoding step returns one credit token for each frame consumed. The credit token count represents the available filtering capacity of the temporal filtering method. When the count is greater than 0, filtering can continue; when the count is equal to 0, filtering pauses and waits for the encoding step to return tokens. The credit token count also represents the available frame data capacity for the encoding step. When the count is less than N, encoding can continue; when the count is equal to N, encoding pauses and waits for the temporal filtering method to consume tokens. The credit token mechanism allows the temporal filtering method and the encoding step to proceed in parallel without alternating pauses, only pausing when tokens are exhausted or full.

[0144] The credit token counter unit is located at the center of the control path between the time-domain filtering method and the encoding step. The input signals to the credit token counter unit are the filtering completion signal sent by the time-domain filtering method and the frame consumption completion signal sent by the encoding step; the output signals are the time-domain filtering pause signal and the encoding step pause signal. Internally, the credit token counter unit contains a 2-bit up / down counter, a frame counter, a comparator group, and pause signal generation logic. The time-domain filtering method sends consumption events to the credit token counter unit via the filtering completion signal line, and the encoding step sends return events to the credit token counter unit via the frame consumption completion signal line. The credit token counter unit sends pause instructions to the time-domain filtering method via the time-domain filtering pause signal line and to the encoding step via the encoding step pause signal line. There is no direct data path between the credit token counter unit and the dynamic random access memory controller (DRAM). The credit token counter unit only controls the progress of the time-domain filtering method and the encoding step through the pause signals; the writing and reading of frame data are still scheduled by the DRAM controller.

[0145] In the credit token mechanism, the signal stream and data stream are separated. The data stream is the transmission path of frame data between the time-domain filtering method, dynamic random access memory, and the encoding step. The signal stream is the transmission path of the filtering completion signal, frame consumption completion signal, and pause signal between the time-domain filtering method, the credit token counting unit, and the encoding step. This separation of data and signal streams allows the credit token counting unit to operate without intervening in the frame data transmission process. It only needs to monitor the occurrence times of the filtering completion event and the frame consumption completion event and generate the pause signal based on the credit token count. The transmission delay of the signal stream is much smaller than that of the data stream, allowing the credit token counting unit to update the credit token count and generate the pause signal within the next clock cycle after the frame data is written or read.

[0146] In the eighth embodiment, writing the filtered frame to the second storage area signifies readiness, and the data source for the encoding step is automatically routed using the high-order bits of the address. This embodiment, based on the eighth embodiment, introduces a credit token counting unit between the time-domain filtering method and the encoding step. The completion of writing the filtered frame to the second storage area triggers a filtering completion signal, and the credit token counting unit executes a consumption event upon receiving the filtering completion signal. After the encoding step reads the filtered frame from the second storage area and completes encoding, it triggers a frame consumption completion signal, and the credit token counting unit executes a return event upon receiving the frame consumption completion signal. The credit token counting unit and the high-order bit automatic routing are independent of each other; the high-order bit automatic routing is responsible for selecting the data source, while the credit token counting unit is responsible for synchronizing the rhythm of the time-domain filtering method and the encoding step; the two work together.

[0147] The credit token mechanism is executed by the credit token counting unit. Located at the center of the control path between the temporal filtering method and the encoding step, the credit token counting unit internally includes a 2-bit up / down counter, a frame counter, a comparator group, and pause signal generation logic. The 2-bit up / down counter maintains the current value of the credit token count, the frame counter maintains the statistical values ​​of the number of filtered and encoded frames, the comparator group determines whether the credit token count is equal to 0 or N, and the pause signal generation logic generates temporal filtering pause signals and encoding step pause signals based on the output of the comparator group. The credit token counting unit is implemented as an independent control module in the hardware video encoder and is connected to the state machines of the temporal filtering method and the encoding step via signal lines.

[0148] The consumption event is triggered when the time-domain filtering method completes filtering of a frame. After completing the filtering process, the filtering module in the time-domain filtering method writes the filtered frame to the second storage area of ​​the dynamic random access memory. Upon completion of the write, the filtering module sends a filtering completion signal to the credit token counter unit via the filtering completion signal line. After receiving the filtering completion signal, the credit token counter unit executes the consumption event on the rising edge of the next clock cycle, decrementing the credit token count by 1. The timing of the consumption event is synchronized with the completion of writing the filtered frame to the second storage area; writing the filtered frame to the second storage area indicates filtering completion, and the filtering completion signal is generated by the write completion flag bit of the filtering module.

[0149] The return event is triggered when a frame is consumed during the encoding step. The encoding step reads the filtered frame from the second memory area of ​​the dynamic random access memory (DRAM), completes intra-frame prediction, inter-frame prediction, transform quantization, and entropy coding, and then sends a frame consumption completion signal to the credit token counter unit via the frame consumption completion signal line. Upon receiving the frame consumption completion signal, the credit token counter unit executes the return event on the rising edge of the next clock cycle, incrementing the credit token count by 1. The timing of the return event is synchronized with the completion of encoding a frame by the encoding step, and the frame consumption completion signal is generated by the encoding step's state machine in the encoding completion state.

[0150] The time-domain filtering pause signal is triggered when the credit token count is 0. When the comparator group of the credit token counting unit detects that the credit token count is equal to 0, the pause signal generation logic immediately sets the time-domain filtering pause signal to an active level. Upon receiving the pause signal, the time-domain filtering method pauses the filtering process. The encoding step pause signal is triggered when the credit token count is N. When the comparator group of the credit token counting unit detects that the credit token count is equal to N, the pause signal generation logic immediately sets the encoding step pause signal to an active level. Upon receiving the pause signal, the encoding step pauses the encoding process. The triggering timing of the pause signal is synchronized with the update time of the credit token count, and the pause signal takes effect in the next clock cycle after the credit token count is updated.

[0151] The credit token counter has a 2-bit width, capable of representing count values ​​from 0 to 3. The total number of credit tokens, N, is set to 2. The 2-bit counter width covers the count range of 0 to N, with a count value of 3 indicating an abnormal state. The 2-bit counter width matches the total number of credit tokens, N=2. The minimum counter width is 2 bits, capable of representing three valid count values ​​(0, 1, 2) and one reserved count value. The choice of a 2-bit counter width minimizes hardware overhead while ensuring coverage of the count range; a 2-bit up / down counter requires only two flip-flops and a small amount of combinational logic.

[0152] The frame counter has a 16-bit width and can represent a count value from 0 to 65535. The frame counter is used to count the number of filtered frames completed by the temporal filtering method and the number of encoded frames completed by the encoding step. The frame counter value is updated when each frame filtering or encoding is completed, and the updated count value is used to generate timing references for the filtering completion signal and frame consumption completion signal. The 16-bit frame counter can cover the frame sequence length at a preset resolution; for a 60fps video sequence, the 16-bit counter can cover approximately 18 minutes of frame sequence.

[0153] The total number of credit tokens, N, is set to 2, indicating that the shared space in the dynamic random access memory can hold 2 frames of data. The temporal filtering method holds all 2 credit tokens at startup, while the encoding step holds 0 credit tokens at startup. The configuration of a total number of credit tokens, N=2, matches the parallel execution requirements of the temporal filtering method and the encoding step. The temporal filtering method can filter 2 frames continuously, and the encoding step can consume 2 frames continuously, with the two processes proceeding in parallel within a 2-frame buffer depth.

[0154] The initial value of the credit token count is set to 2, meaning the temporal filtering method holds all credit tokens upon startup. An initial value of 2 indicates that the temporal filtering method can begin filtering immediately without waiting for the encoding step to return tokens. The encoding step waits for the temporal filtering method to consume tokens upon startup; when the credit token count is less than 2, the encoding step begins consuming frame data. The initial count configuration matches the startup timing of the temporal filtering method, which enters filtering mode immediately after the hardware video encoder starts.

[0155] The consumption event is triggered when the time-domain filtering method completes filtering for one frame, decrementing the credit token count by 1. After receiving the filtering completion signal, the credit token counting unit performs a subtraction operation on the rising edge of the next clock cycle, decrementing the current value of the credit token count by 1. When the credit token count is 2, the count becomes 1 after the consumption event; when the credit token count is 1, the count becomes 0; when the credit token count is 0, the consumption event is not executed, and the time-domain filtering method pauses filtering. The count transformation of the consumption event is implemented by a 2-bit up / down counter, and the subtraction operation is executed when the credit token count is greater than 0.

[0156] The return event is triggered when a frame is consumed during the encoding step, and the credit token count is incremented by 1. After receiving the frame consumption completion signal, the credit token counting unit performs an addition operation on the rising edge of the next clock cycle, incrementing the current value of the credit token count by 1. When the credit token count is 0, the count becomes 1 after the return event; when the credit token count is 1, the count becomes 2 after the return event; when the credit token count is 2, the return event is not executed, and the encoding step pauses encoding processing. The count transformation of the return event is implemented by a 2-bit up / down counter, and the addition operation is executed when the credit token count is less than N.

[0157] The determination of a zero count for the credit token is implemented by a comparator group. The comparator group compares the output value of a 2-bit up / down counter with a constant 0. When the counter output value equals 0, the comparator group outputs a zero count flag. This zero count flag is input to the pause signal generation logic, which sets the time-domain filtered pause signal to an active level. Upon receiving the pause signal, the time-domain filtering method pauses the filtering process and waits for the credit token to be returned during the encoding step. The determination of a zero count is completed within the next clock cycle after the credit token count is updated, with a determination delay of one clock cycle.

[0158] The determination of whether the credit token count is N is implemented by a comparator group. The comparator group compares the output value of a 2-bit up / down counter with a constant 2. When the counter output value equals 2, the comparator group outputs a count of N flag. The count of N flag is input to the pause signal generation logic, which sets the pause signal of the encoding step to an active level. Upon receiving the pause signal, the encoding step pauses the encoding process and waits for the time-domain filtering method to consume the credit token. The determination of the count of N is completed within the next clock cycle after the credit token count is updated, with a determination delay of 1 clock cycle.

[0159] Under the credit token mechanism, the frame period is the maximum value of the filtering duration and the encoding duration. The frame period formula is introduced as follows: ,in For frame periods under the credit token mechanism, The duration of a frame processed by the temporal filtering method. The frame period is the time taken to process one frame in the encoding step. Physically, this means that the temporal filtering method and the encoding step proceed in parallel, with the frame period determined by the slower of the two. Taking a 60fps frame rate as an example, with a frame period constraint of 16.67 milliseconds, the temporal filtering method processes one frame in 12 milliseconds, the encoding step processes one frame in 14 milliseconds, and the frame period under the trust token mechanism is... The time interval is less than 16.67 milliseconds, indicating timing convergence. The frame period for the ping-pong alternating pauses is... For frames longer than 16.67 milliseconds, timing does not converge. The trust token mechanism reduces frame period by approximately 46% and increases throughput by approximately 85.7%.

[0160] The credit token counter register is a 2-bit up / down counter that stores the current value of the credit token count. The register's initial value is 2. The count is decremented by 1 when a consumption event is triggered and incremented by 1 when a return event is triggered. The output of the counter register is input to the comparator group, which generates a 0-count flag and a N-count flag based on the register's output. The counter register has a 2-bit width, matching the total number of credit tokens N=2, and its count range is 0 to 3. A count of 3 indicates an abnormal state.

[0161] The frame counter register is a 16-bit counter that stores statistical values ​​of the number of filtered frames and the number of encoded frames. The frame counter increments by 1 when a frame is filtered using the temporal filtering method and when a frame is encoded using the encoding method. The output value of the frame counter is used as a timing reference for generating the filtering completion signal and the frame consumption completion signal, and is also used for frame index management of the frame sequence. The frame counter has a 16-bit width, which can cover the frame sequence length at a preset resolution.

[0162] The temporal filtering method and the encoding step share a storage space capacity of 2 frames in the dynamic random access memory (DRAM). The temporal filtering method writes the filtered frames into the second storage area, and the encoding step reads the filtered frames from the second storage area. The capacity of the second storage area can hold 2 frames of filtered frame data. The total number of trust tokens, N=2, matches the shared space capacity of 2 frames. The temporal filtering method can write up to 2 frames of filtered frames in advance, and the encoding step can consume up to 2 frames of filtered frames with a delay. The allocation of the shared space is completed during the initialization phase of the DRAM controller, and the controller configures the boundary addresses of the shared space according to the preset resolution.

[0163] The credit token counter register and the frame counter register work together. The frame counter records the cumulative number of filtered and encoded frames, while the credit token counter register records the currently available filtering quota. The frame counter is updated upon completion of each frame filtering or encoding, while the credit token counter register is updated when a consumption event or return event is triggered. The updates to the frame counter and the credit token counter register are completed within the same clock cycle. The frame counter's count value is used as a timing reference for generating the filtering completion signal and the frame consumption completion signal, while the credit token counter register's count value is used to generate the pause signal.

[0164] The time-domain filtering pause signal generated by the credit token counting unit is sent to the state machine of the time-domain filtering method via a dedicated signal line. Upon receiving the pause signal, the state machine of the time-domain filtering method pauses the filtering process, maintaining the current filtering state. During the pause, the time-domain filtering method does not initiate new filtering requests, write filtering frames, or consume credit tokens. The time-domain filtering pause signal is a level signal; the active level is high, set to active when the credit token count is 0, and set to inactive when the credit token count is greater than 0.

[0165] The encoding step pause signal generated by the credit token counting unit is sent to the encoding step's state machine via a dedicated signal line. Upon receiving the pause signal, the encoding step's state machine pauses encoding processing and maintains its current encoding state. During the pause, the encoding step does not initiate new data retrieval requests, consume frame data, or return credit tokens. The encoding step pause signal is a level signal; the active level is high, set to active when the credit token count is N, and set to inactive when the credit token count is less than N.

[0166] The credit token mechanism features bidirectional pauses with zero additional handshake signals. Both the time-domain filtering pause signal and the encoding step pause signal are directly generated by the credit token counting unit based on the credit token count, eliminating the need for additional handshake interactions between the time-domain filtering method and the encoding step. After pausing, the time-domain filtering method waits for the encoding step to return the credit token; conversely, after pausing, the encoding step waits for the time-domain filtering method to consume the credit token. This waiting process requires no additional handshake signals; only changes in the credit token count need to be monitored. Compared to the ping-pong dual-signal alternating pause, the credit token mechanism eliminates the need for additional handshake signal lines, simplifying control logic and reducing pause response latency.

[0167] The pause signal generated by the credit token counter is not directly sent to the dynamic random access memory (DRAM) controller. The DRAM controller schedules memory operations based on read / write requests from the time-domain filtering method and the encoding step. The pause signal indirectly affects the initiation of read / write requests through the state machines of the time-domain filtering method and the encoding step. No write request is initiated after the time-domain filtering method pauses, and no read request is initiated after the encoding step pauses, thus idling the DRAM controller's read / write scheduling. There is no direct signal connection between the credit token counter and the DRAM controller, avoiding cross-coupling of control paths.

[0168] In response to the abnormal situation where the credit token counter register has a count value of 3, this embodiment adopts the conventional default configuration. The credit token counter unit regards the count value of 3 as an abnormal state. The pause signal generation logic simultaneously sets the time domain filtering pause signal and the encoding step pause signal to the active level. Both the time domain filtering method and the encoding step are paused and wait for the reset signal to clear the abnormal state to ensure the integrity of the process.

[0169] In response to the abnormal situation of loss of filtering completion signal or frame consumption completion signal, this embodiment adopts the conventional default configuration. When the frame counter count value and the update timing of the credit token count register are inconsistent, the credit token counting unit checks whether the update of the credit token count register is complete by checking the frame counter count value. If the update is detected to be missing, the credit token counting unit resets the credit token count register to the initial value 2 to ensure the integrity of the process.

[0170] In response to the abnormal situation where the pause time of the time-domain filtering method or encoding step exceeds the preset threshold, this embodiment adopts the conventional default configuration. The credit token counting unit starts the timeout counter during the valid pause signal. When the timeout counter count value exceeds the preset threshold, the credit token counting unit outputs an abnormal alarm signal. The control processor of the hardware video encoder performs a reset operation according to the alarm signal to ensure the integrity of the process.

[0171] To address the abnormal situation where the credit token counter register saturates under continuous return events, this embodiment adopts the conventional default configuration. When the credit token count equals N, the credit token counter unit ignores subsequent return events, the encoding step pause signal remains at a valid level, and the encoding step pauses the encoding process to ensure the integrity of the process.

[0172] In the above process, the credit token counting unit is configured with 2 credit tokens, the credit token counting bit width is 2 bits, the frame counter bit width is 16 bits, the temporal filtering method consumes 1 credit token for each frame filtering completed, and the encoding step returns 1 credit token for each frame consumed. Temporal filtering is paused when the credit token count is 0, and the encoding step is paused when the credit token count is N. The credit token counting achieves pipeline synchronization without pauses, and the temporal filtering method and the encoding step proceed in parallel. The frame period is the maximum value of the filtering duration and the encoding duration, avoiding buffer overflow and data conflicts. Bidirectional pauses with zero additional handshake signals improve the throughput and timing convergence of the hardware video encoder.

[0173] Tenth Implementation Method In the ninth embodiment, bidirectional flow control using credit tokens synchronizes the temporal filtering method with the encoding step. The temporal filtering method holds N credit tokens, consuming one credit token for each frame filtered. The encoding step returns one credit token for each frame consumed. Token counting enables pipeline synchronization without pauses, avoiding buffer overflows and data conflicts. The frame period is the maximum of the filtering duration and the encoding duration, improving hardware utilization and timing convergence. The inventors of this invention discovered that if the filtering intensity in the temporal filtering method is determined by a fixed setting or only by simple criteria, noise removal is insufficient in flat areas with large quantization errors. Residual noise accumulates in the temporal domain, causing flickering artifacts in the reconstructed image. In areas with rich texture, it is easy to over-smooth and lose details, and the reduced edge and texture sharpness affects subjective quality. The tenth embodiment of the present invention, based on the aforementioned embodiments, proposes to obtain the quantization error energy and frequency domain distribution corresponding to the current frame to be filtered, and to adaptively determine the filtering intensity based on the quantization error energy and frequency domain distribution. Strong filtering is used to remove noise in areas with large quantization errors and flat areas, while weak filtering or bypassing is used to preserve texture details in areas with small quantization errors or rich textures. The filtering intensity prediction and rate-distortion optimization decision are carried out in parallel without increasing the critical path delay. Figure 10 The diagram illustrates the joint decision-making process of quantization error energy and frequency domain distribution. The left side of the diagram shows the quantization error energy calculation path. The signals from the output of the transform module and the output of the dequantizer are input into a tree-shaped accumulator after passing through a subtractor and a multiplier. The middle of the diagram shows the frequency domain distribution statistical path. The quantization error coefficients are input into three accumulators (low frequency, mid frequency, and high frequency) according to frequency band. The right side of the diagram shows the joint decision-making module. The quantization error energy and the frequency domain distribution statistical values ​​are input into the filter strength decision-making logic, and the filter strength is output to the filter processing module.

[0174] Quantization error energy represents the degree of reconstruction loss in the transform domain coefficients during quantization and dequantization. In video coding, the transform module transforms pixel residuals into frequency domain coefficients, the quantizer performs lossy quantization on the frequency domain coefficients, and the dequantizer reconstructs the quantized coefficients into approximate frequency domain coefficients. The difference between the original frequency domain coefficients and the reconstructed frequency domain coefficients is the quantization error. The quantization error energy is the sum of the squares of this difference, reflecting the total amount of frequency domain information lost during the quantization process. The larger the quantization error energy, the more severe the quantization loss, the greater the frequency domain difference between the reconstructed image and the original image, and the more powerful the temporal filtering needs to suppress the noise introduced by quantization. Quantization error energy differs from the rate-distortion value in the pixel domain. The rate-distortion value is based on the difference between the original pixel and the reconstructed pixel in the pixel domain, while the quantization error energy is based on the difference between the original transform coefficients and the dequantized reconstructed coefficients in the transform domain. The two are different in both numerical source and physical meaning.

[0175] Frequency domain distribution represents the proportion of quantization error energy distributed across the low, mid, and high frequency bands. In image content, the energy of flat regions is concentrated in the low-frequency band, while the energy of textured regions is distributed in the mid and high frequency bands. The frequency domain distribution of quantization error follows the same pattern: the quantization error energy of flat regions is concentrated in the low-frequency band, while the quantization error energy of textured regions is distributed in the mid and high frequency bands. Frequency domain distribution is characterized by the statistical proportion of quantization error energy in each frequency band. A low proportion of energy in the high-frequency band indicates that the current block is a flat region, while a high proportion indicates that the current block is a textured region. Frequency domain distribution provides information about the image content dimension for determining the filter intensity, enabling the filter intensity to be adaptively adjusted according to the image content.

[0176] The topology of the quantization error energy calculation path is as follows: the output of the transformation module provides the original transformation coefficients. The quantizer output provides the quantized coefficients. The output of the dequantizer provides the dequantization reconstruction coefficients. Original transformation coefficients With inverse quantization reconstruction coefficients Input subtractor calculates quantization error Quantization error Input multiplier to calculate square value The squared values ​​are input into a tree accumulator and accumulated level by level to obtain the quantization error energy. The topology of the frequency domain distribution statistical path is as follows: quantization error Simultaneously, the frequency band division logic is input. Based on the coefficient coordinates (u, v), the frequency band division logic determines the corresponding frequency band and accumulates the squared quantization error value to the low-frequency energy accumulator, mid-frequency energy accumulator, and high-frequency energy accumulator respectively. The output values ​​of the three accumulators are then input into the frequency domain distribution calculation logic to obtain the high-frequency energy proportion. Quantization error energy With high frequency energy ratio The common input is the filter strength decision logic, which outputs the filter strength S to the filter processing module. Data is transmitted between modules via dedicated signal lines. The subtractor output is connected to the multiplier input via a 16-bit signal line, the multiplier output is connected to the tree accumulator input via a 32-bit signal line, the tree accumulator output is connected to the filter strength decision logic input via a 16-bit signal line, and the filter strength decision logic output is connected to the filter processing module control terminal via a 2-bit signal line.

[0177] In the aforementioned ninth embodiment, the credit token counting unit is responsible for synchronizing the temporal filtering method with the encoding step. Writing a filtered frame to the second storage area triggers a filtering completion signal, and the encoding step consuming frame data triggers a frame consumption completion signal. This embodiment, based on the ninth embodiment, adds a quantization error energy calculation unit, a frequency domain distribution statistics unit, and a filter strength decision unit within the temporal filtering method. The quantization error energy calculation unit and the frequency domain distribution statistics unit extract signals from the output of the transform module and the dequantizer, and are not involved in the storage and transmission of frame data; their data paths are independent of the credit token counting unit. The filter strength S output by the filter strength decision unit is used by the filtering processing module. After completing filtering, the filtering processing module still sends a filtering completion signal to the credit token counting unit through the filtering completion signal line, and the flow control logic of the credit token mechanism remains unchanged.

[0178] The execution unit for the joint decision-making of quantization error energy and frequency domain distribution includes a quantization error energy calculation unit, a frequency domain distribution statistics unit, and a filter strength decision unit. The quantization error energy calculation unit is located between the output of the transform module and the output of the dequantizer, and internally contains a subtractor group, a multiplier group, and a tree-structured accumulator. The frequency domain distribution statistics unit is located in the parallel branch of the quantization error energy calculation unit, and internally contains frequency band division logic, three frequency band energy accumulators, and a divider. The filter strength decision unit is located at the output of the quantization error energy calculation unit and the frequency domain distribution statistics unit, and internally contains a lookup table ROM and boundary correction combinational logic. These three units are implemented as independent combinational logic modules in the hardware video encoder and are connected to the state machine of the time-domain filtering method via an enable signal line.

[0179] Quantization error energy calculation outputs the original transformation coefficients in the transformation module. Furthermore, the dequantizer outputs dequantization reconstruction coefficients. Triggered at time. After the transformation module completes the transformation process, the original transformation coefficients... Once the output of the transform module is stable, the quantizer performs quantization on the original transform coefficients, and the dequantizer performs dequantization on the quantized coefficients to reconstruct the coefficients. The output of the dequantizer is stable. The quantization error energy calculation unit starts when the enable signal is valid. The enable signal is generated by the state machine of the time-domain filtering method when the transformation module outputs a valid flag. The quantization error energy calculation unit is a pure combinational logic that completes the subtraction, squaring, and accumulation operations of all coefficient pairs within 2 to 3 clock cycles after the enable signal is valid, and outputs the quantization error energy. .

[0180] Frequency domain distribution statistics and quantization error energy calculation are triggered in parallel. The frequency domain distribution statistics unit starts when the same enable signal is active, and begins processing simultaneously with the quantization error energy calculation unit. The frequency domain distribution statistics unit receives the quantization error... As input, the band division logic determines the corresponding frequency band based on the coefficient coordinates (u,v), and accumulates the squared quantization error values ​​to the energy accumulator of the corresponding frequency band. The frequency domain distribution statistics unit is a pure combinational logic that completes the band division and accumulation operations for all coefficient pairs within 2 to 3 clock cycles after the enable signal is active, outputting the high-frequency energy percentage. The quantization error energy calculation unit and the frequency domain distribution statistics unit work in parallel, with both having a processing delay of 2 to 3 clock cycles and completing at the same time.

[0181] Filter strength prediction in quantization error energy With high frequency energy ratio Triggered upon stable output. Output of the quantization error energy calculation unit. Then, the frequency domain distribution statistical unit outputs... Then, the filter strength decision unit initiates a table lookup operation. The filter strength decision unit is combinational logic; the table lookup operation is completed within 1 to 2 clock cycles, outputting the filter strength S. After the filter strength S is stably output, the filter strength decision unit generates... The signal is sent to the filtering module, which initiates filtering preparation based on the filtering intensity S. The completion time of the filtering intensity prediction is earlier than the completion time of the rate-distortion optimization decision; both proceed in parallel, and the filtering intensity prediction does not occupy the critical path of the rate-distortion optimization decision.

[0182] The filter strength prediction and rate-distortion optimization decision are performed in parallel over time. The completion time of the rate-distortion optimization decision is... ,in The number of cycles processed by the transformation and quantization module, The inverse quantization and inverse transform module processes the number of cycles. To calculate the number of cycles for rate distortion, To optimize the number of decision cycles for rate distortion, the completion time for filter strength prediction is... ,in The number of cycles is used to calculate the quantization error energy. The number of statistical periods for frequency domain distribution. The number of lookup cycles is used to determine the filter strength. Taking a 16x16 block as an example... The cycle is 16 to 32. The cycle is 8 to 16. It lasts for 4 to 8 cycles. For 2 to 3 cycles, For 2 to 3 cycles, For 1 to 2 cycles, and The difference is = 25 to 53 cycles, worst case Take 16, Take 8 Take 4. Take 3, Take 3, Taking 2, the difference is still greater than 0 for 23 cycles. The completion time of the filter intensity prediction is at least 25 cycles earlier than the completion time of the rate distortion optimization decision. The filter processing module can start filter preparation before the rate distortion optimization decision is completed, without increasing the critical path delay.

[0183] Quantization error signal The bit width is 16 bits for signed numbers. Original transform coefficients. 16-bit signed number, dequantized reconstruction coefficients The quantization error is obtained by subtracting the two values ​​from a 16-bit signed number. It is a 16-bit signed number with a value range of -32768 to 32767. The 16-bit width can cover the dynamic range of the transform coefficients, and the magnitude of the quantization error does not exceed twice the larger of the absolute values ​​of the original transform coefficients and the dequantized reconstructed coefficients. The 16-bit signed number can represent this range. The quantization error signal is generated at the output of the subtractor and transmitted to the input of the multiplier and the frequency band division logic input through the 16-bit signal line.

[0184] Quantization error energy It is a 16-bit unsigned number. Quantization error energy. ,in It is a 16-bit signed number. The result is a 32-bit unsigned number. The sum of the squares of all coefficient pairs results in a 48-bit unsigned number. The 48-bit sum is truncated down to the lower 16 bits, retaining the higher 16 bits as quantization error energy. ,Right now A 16-bit unsigned number ranges from 0 to 65535, representing the relative magnitude of quantization error energy. The truncation operation discards the lower 16 bits of precision, retaining the higher 16 bits of energy, thus meeting the precision requirements for filter strength determination. Taking an 8x8 block as an example, after summing the squares of the 64 coefficient pairs, the higher 16 bits of energy value can distinguish the energy differences under different quantization parameters.

[0185] Frequency band assignment is determined by the sum of the coefficient coordinates (u, v). For an 8x8 transform block, the low-frequency band (LF) has u+v ≤ 4 and contains 15 coefficients; the mid-frequency band (MF) has 5 ≤ ​​u+v ≤ 8 and contains 28 coefficients; and the high-frequency band (HF) has u+v ≥ 9 and contains 21 coefficients. For a 16x16 transform block, the low-frequency band (LF) has u+v ≤ 8 and contains 45 coefficients; the mid-frequency band (MF) has 9 ≤ u+v ≤ 16 and contains 91 coefficients; and the high-frequency band (HF) has u+v ≥ 17 and contains 120 coefficients. For a 32x32 transform block, the low-frequency band (LF) has u+v ≤ 16 and contains 153 coefficients; the mid-frequency band (MF) has 17 ≤ u+v ≤ 32 and contains 375 coefficients; and the high-frequency band (HF) has u+v ≥ 33 and contains 496 coefficients. The frequency band assignment threshold scales linearly with the block dimension. Where N is the block size, with a threshold of 4 for 8x8 blocks, 8 for 16x16 blocks, and 16 for 32x32 blocks. The scaling rule ensures that the low-frequency band covers approximately one-quarter of the coefficient space, the mid-frequency band covers approximately half, and the high-frequency band covers approximately one-quarter, while maintaining consistency in the physical semantics of each frequency band across different block sizes.

[0186] Frequency domain distribution statistics It is an 8-bit unsigned number. ,in This represents the accumulated energy value in the high-frequency band. This is the cumulative energy value across all frequency bands. It is an 8-bit fixed-point number with a value range of 0 to 255, corresponding to a high-frequency energy percentage of 0% to 100%. The 8-bit width can distinguish 256 energy percentage levels, meeting the accuracy requirements of filter intensity determination. A higher value indicates a higher proportion of high-frequency energy, resulting in richer block texture. The smaller the value, the lower the proportion of high-frequency energy and the flatter the current block.

[0187] Quantization error is calculated using a subtractor. The quantization error calculation formula is introduced as follows: = - ,in For quantization error, These are the original transformation coefficients at the output of the transformation module. These are the dequantization reconstruction coefficients at the output of the dequantizer. Physically, they represent the loss caused to the frequency domain coefficients by the quantization and dequantization processes. A positive value indicates that the original coefficients are greater than the reconstructed coefficients, while a negative value indicates that the original coefficients are less than the reconstructed coefficients. Taking an 8x8 block as an example, 64 coefficients are input in parallel to 64 sets of subtractors. Each subtractor performs a 16-bit signed subtraction operation and outputs a 16-bit signed quantization error. The subtractors are purely combinational logic with no clock delay, and the output is valid for one logic gate delay after the inputs stabilize.

[0188] Quantization error energy calculation is implemented using a multiplier and a tree-structured accumulator. The formula for quantization error energy calculation is introduced as follows: ,in Here, N represents the quantization error energy, and N is the transform block size. This refers to the quantization error. Physically, it represents the sum of squares of the quantization errors, reflecting the total loss of frequency domain information during the quantization process. In hardware implementation, each quantization error... Input the corresponding multiplier to calculate the square value. The multiplier performs 16-bit signed multiplication and outputs a 32-bit unsigned square value. All square values ​​are input into a tree accumulator, which is a 48-bit adder array. The first stage adds adjacent square values ​​pairwise, the second stage adds the results from the first stage pairwise, and so on, until a 48-bit accumulated result is obtained after log2(N²) stages of addition. Taking an 8x8 block as an example, 64 square values ​​are accumulated through 6 stages of tree accumulation to obtain a 48-bit accumulated result. The accumulated result is truncated to the lower 16 bits, retaining the higher 16 bits as quantization error energy. The tree accumulator is a pure combinational logic, with each level of addition executed in parallel. The total latency is the same as that of a 6-level adder, approximately 2 to 3 clock cycles.

[0189] Frequency domain distribution statistics are implemented using frequency band partitioning logic and a frequency band energy accumulator. Quantization error. The input frequency band division logic determines the frequency band based on the sum of the coefficient coordinates (u, v), and then calculates the squared quantization error value. The values ​​are respectively accumulated to the low-frequency, mid-frequency, and high-frequency energy accumulators. Each of the three frequency band energy accumulators is a 48-bit adder, accumulating the squared value of its corresponding frequency band. After all coefficients are processed, the high-frequency energy accumulation value is... Total energy across all frequency bands Input divider, divider calculation It outputs an 8-bit fixed-point number. The divider is implemented using combinational logic and employs a shift-and-subtract algorithm, with a delay of approximately 1 to 2 clock cycles. The frequency domain distribution statistics unit and the quantization error energy calculation unit operate in parallel, and both complete at the same time.

[0190] The filter strength decision is implemented through a lookup table ROM and boundary correction combinational logic. The filter strength decision unit receives the quantization error energy. With high frequency energy ratio As input, Quantized to 4-bit level, Quantization is performed at 4-bit levels, and combination is represented by an 8-bit address. A lookup table ROM stores 48 filter strength configuration entries, each a 2-bit filter strength value ranging from 0 to 3. 0 indicates bypass, 1 indicates weak filtering, 2 indicates medium filtering, and 3 indicates strong filtering. The lookup table rules are: output filter strength 3 when quantization error energy is large and high-frequency energy percentage is low; output filter strength 1 when quantization error energy is large and high-frequency energy percentage is high; output filter strength 1 when quantization error energy is small and high-frequency energy percentage is low; and output filter strength 0 when quantization error energy is small and high-frequency energy percentage is high. Boundary correction combinational logic corrects the lookup table results. When the quantization error energy exceeds a preset upper limit, the filter strength is limited to 3; when the high-frequency energy percentage exceeds a preset upper limit, the filter strength is limited to 0 or 1. The filter strength decision unit is combinational logic, the lookup operation has a delay of 1 to 2 clock cycles, and the output filter strength S is a 2-bit unsigned number.

[0191] The parallel relationship between filter intensity prediction and rate-distortion optimization decision is anchored by a time-series formula. The formula for the completion time of filter intensity prediction is introduced as follows: ,in To predict the completion time of the filter intensity, The number of cycles processed by the transformation and quantization module, The number of cycles is used to calculate the quantization error energy. The number of statistical periods for frequency domain distribution. The number of decision cycles is determined by looking up a table for the filter intensity. The formula for optimizing the decision completion time by introducing rate distortion is as follows: ,in To optimize the decision completion time to reduce distortion, The inverse quantization and inverse transform module processes the number of cycles. To calculate the number of cycles for rate distortion, The number of decision cycles for rate-distortion optimization is specified. Physically, this means that filter strength prediction and rate-distortion optimization decision-making share the transform and quantization stages. Filter strength prediction proceeds in parallel during the inverse quantization and inverse transform stages, while rate-distortion optimization decision-making requires rate-distortion calculation and decision-making after inverse quantization and inverse transform are completed. Taking a 16x16 block as an example... The cycle is 16 to 32. The cycle is 8 to 16. It lasts for 4 to 8 cycles. For 2 to 3 cycles, For 2 to 3 cycles, For 1 to 2 cycles, and The difference is = 25 to 53 cycles, and in the worst case, the difference is still greater than 0 for 23 cycles. The completion time of the filter intensity prediction is earlier than the completion time of the rate distortion optimization decision. The filter processing module can start the filter preparation before the rate distortion optimization decision is completed, without increasing the critical path delay.

[0192] The quantization error energy calculation unit is internally configured with a 48-bit accumulator to accumulate the squared values ​​of all coefficient pairs. The accumulator has a tree structure: the first stage contains N² / 2 48-bit adders, the second stage contains N² / 4 48-bit adders, and so on, with the last stage containing one 48-bit adder. Taking an 8x8 block as an example, the first stage contains 32 48-bit adders, the second stage contains 16, the third stage contains 8, the fourth stage contains 4, the fifth stage contains 2, and the sixth stage contains 1, for a total of 63 48-bit adders. Intermediate results are temporarily stored in the output registers of each adder stage, and the final result is temporarily stored in the output register of the last adder stage. The accumulator has a 48-bit width, covering the accumulation range of 64 32-bit squared values, with a maximum accumulation value of [value missing]. 48-bit width is sufficient.

[0193] Quantization error energy The result is temporarily stored in a 16-bit energy register. The 48-bit output of the tree accumulator is truncated (lower 16 bits) and the higher 16 bits are written to the energy register. The energy register has a bit width of 16 bits, consistent with the output bit width of the quantization error energy. The energy register is updated when the quantization error energy calculation unit outputs a valid flag, and the updated value is transmitted to the filter strength decision unit via a 16-bit signal line. The output of the energy register remains stable until the next enable signal is valid, providing a stable input to the filter strength decision unit.

[0194] The frequency domain distribution statistics unit is internally configured with three 48-bit frequency band energy registers, which temporarily store the accumulated energy values ​​of the low-frequency band, mid-frequency band, and high-frequency band, respectively. These three frequency band energy registers are accumulated sequentially during the frequency domain distribution statistics unit's processing. After all coefficients have been processed, the accumulated energy value of the high-frequency band and the accumulated energy values ​​of all frequency bands are input into a divider to calculate the proportion of high-frequency energy. The bandwidth of the band energy register is 48 bits, which is the same as the output bandwidth of the band energy accumulator. The output of the band energy register remains stable after the divider calculation, providing a stable input to the filter strength decision unit.

[0195] The filter strength S is temporarily stored in a 2-bit filter strength register. The 2-bit filter strength value output by the filter strength decision unit is written into the filter strength register, and the filter strength register is... The filter strength register is updated when the signal is valid. It has a 2-bit width and can represent filter strength levels from 0 to 3. The output of the filter strength register is transmitted to the control terminal of the filter processing module via a 2-bit signal line. The filter processing module configures the filter parameters based on the filter strength value. The output of the filter strength register remains stable until the filter strength prediction for the next block is completed.

[0196] The filter strength S output by the filter strength decision unit is a 2-bit unsigned number, ranging from 0 to 3. The filter strength S is transmitted to the control terminal of the filter processing module via a 2-bit signal line. The filter processing module configures the filter parameters according to the filter strength S: bypass filtering is applied when the filter strength is 0, weak filtering parameters are used when the filter strength is 1, medium filtering parameters are used when the filter strength is 2, and strong filtering parameters are used when the filter strength is 3. The value of the filter strength S is within... Once the signal is valid, it remains stable, and the filtering module receives it. After the signal is received, the filter strength S is read and filtering preparation is initiated.

[0197] The filter strength decision unit generates the filter strength decision unit after the filter strength S is stabilized. Signal. The signal is a pulse signal, with an effective level of high and a pulse width of one clock cycle. The signal is transmitted to the control terminal of the filtering module via a dedicated signal line. Upon receiving the signal, the filtering module... After receiving the signal, configure the filtering parameters according to the filtering strength S and start filtering preparation. The signal is generated at the time when the filter intensity prediction is completed, which is at least 25 cycles earlier than the time when the rate distortion optimization decision is completed. The filter processing module can start filter preparation before the rate distortion optimization decision is completed.

[0198] The filtering module receives After the signal is received, filtering preparation begins. The filtering module reads the filter strength S and configures the filtering parameters based on S, including the filter window size, filter coefficients, and filter threshold. When the filter strength is 0, the filtering module bypasses the filtering process; when the filter strength is 1, it configures weak filtering parameters for a small window; when the filter strength is 2, it configures filtering parameters for a medium window; and when the filter strength is 3, it configures strong filtering parameters for a large window. After filtering preparation is complete, the filtering module performs filtering on the current frame to be filtered. After filtering is complete, the filtered frame is written to the second storage area of ​​the dynamic random access memory, and a filtering completion signal is sent to the credit token counter unit via the filtering completion signal line.

[0199] The filter strength prediction and rate-distortion optimization decisions are performed in parallel, and their outputs are independent of each other. The output of the filter strength prediction is the filter strength S and... The output of the rate-distortion optimization decision is the block partitioning mode and coding parameters. The filter strength S is already stably output before the rate-distortion optimization decision is completed, and the filtering preparation module has started filtering preparation before the rate-distortion optimization decision is completed. After the rate-distortion optimization decision is completed, the coding step performs coding processing according to the decision result. The filtering preparation of the filtering processing module and the coding processing of the coding step proceed in parallel, without increasing the critical path delay.

[0200] To address the anomaly of 48-bit accumulator overflow during the quantization error energy accumulation process, this embodiment adopts the conventional default configuration. When the tree accumulator generates a carry in the highest bit, it outputs a saturation flag. When the saturation flag is valid, the accumulation result is clamped to the maximum value of 48 bits. After truncating the high 16 bits, the energy register is written with 65535. The filter strength decision unit outputs a filter strength of 3 according to the energy upper limit to ensure the integrity of the process.

[0201] To address the anomaly where the number of coefficients within a certain frequency band is zero after frequency band division, this embodiment employs the conventional default configuration: the frequency band energy accumulator outputs 0, the divider outputs 0 when the denominator is zero, and the high-frequency energy percentage... If the value is 0, the filter strength decision unit outputs the filter strength based on the ratio of quantization error energy to high-frequency energy being 0, to ensure the integrity of the process.

[0202] In response to the abnormal situation where the lookup table ROM of the filter strength decision unit encounters an undefined address, this embodiment adopts the conventional default configuration. The lookup table ROM outputs a filter strength of 1, and the filter processing module performs filter processing according to the weak filter parameters to ensure the integrity of the process.

[0203] To address the anomaly where the completion time of filter intensity prediction lags behind the completion time of rate-distortion optimization decision, this embodiment employs the conventional default configuration. The filter processing module receives... After the signal is received, filtering preparation begins. If rate-distortion optimization decision has been completed, then... Signal not arrived, filtering module waits The signal waits for a period of time and does not initiate a filtering request to ensure the integrity of the process.

[0204] In the above process, the quantization error energy calculation unit extracts the original transformation coefficients from the output of the transformation module and the output of the dequantizer. With inverse quantization reconstruction coefficients The quantization error is calculated using a subtractor. The square value is calculated using a multiplier. The quantization error energy is obtained by accumulating all squared values ​​using a tree accumulator and truncating the high 16 bits. The quantization error energy is a 16-bit unsigned number. The frequency domain distribution statistics unit and the quantization error energy calculation unit work in parallel. The squared quantization error value is accumulated to the low-frequency, mid-frequency, and high-frequency energy accumulators through frequency band division logic. The high-frequency energy proportion is calculated using a divider. The filter strength decision unit determines the filter strength based on the quantization error energy. With high frequency energy ratio The filter intensity S is determined by a joint lookup table. Strong filtering is used to remove noise in areas with large quantization errors and flat surfaces, while weak filtering or bypassing is used to preserve texture details in areas with small quantization errors or rich textures. The filter intensity prediction is completed at least 25 cycles earlier than the rate-distortion optimization decision. The filtering module starts filtering preparation before the rate-distortion optimization decision is completed, without increasing critical path latency. The filtering module performs filtering on the current frame to be filtered using the filter intensity S. After filtering, the filtered frame is written to the second storage area. This works in conjunction with bidirectional flow control using a credit token to improve the noise suppression and detail preservation capabilities of the hardware video encoder.

[0205] Eleventh Implementation Method The eleventh embodiment of the present invention also provides a computer program product or a computer-readable storage medium having a computer program. When executed by a processor, the computer program implements the steps of the time-domain filtering methods in the above embodiments.

[0206] Finally, it should be noted that those skilled in the art will understand that many technical details have been presented in the embodiments of the present invention to facilitate a better understanding of the invention. However, even without these technical details and various variations and modifications based on the above embodiments, the technical solutions claimed in the claims of the present invention can be substantially achieved. Therefore, in practical applications, various changes in form and detail can be made to the above embodiments without departing from the spirit and scope of the present invention.

Claims

1. A temporal filtering method for a hardware video encoder in the preprocessing workflow, characterized in that, include: Obtain the current frame to be filtered; The coarse motion estimation module, the fine-tuned motion estimation module, and the fractional motion estimation module in the hardware video encoder are reused to perform the downsampling and motion search required for motion-compensated temporal filtering on the current frame to be filtered; The rate-distortion optimization module in the hardware video encoder is reused to perform the interpolation and motion compensation required for motion-compensated temporal filtering on the current frame to be filtered. Perform filtering processing on the current frame to be filtered to obtain a filtered frame; The filtered frame is stored in a dynamic random access memory as an input frame for encoding.

2. The method according to claim 1, characterized in that, The step of performing filtering processing on the current frame to be filtered to obtain a filtered frame includes: Obtain the original pixels, interpolated optimal inter-frame predicted pixels, block partitioning results, and quantization parameters of the current coding block in the current frame to be filtered; Based on the original pixels, the interpolated optimal inter-frame predicted pixels, the block partitioning result, and the quantization parameters, a bilateral filter is used to perform filtering processing on the current frame to be filtered, resulting in filtered pixels. The filtered pixels are stored as the filtered frame in the dynamic random access memory.

3. The method according to claim 1, characterized in that, The method further includes: Motion prediction and rate-distortion optimization decisions are performed on the 8x8, 16x16 and 32x32 coded blocks in the current frame to be filtered, and the optimal block is divided into corresponding prediction blocks as the input of the filtering process. The transform used in the rate-distortion optimization decision is a discrete cosine transform, a discrete sine transform, or a Hadamard transform.

4. The method according to claim 1, characterized in that, The method further includes: The frame sequence is organized into image groups of 4 frames each. The frame types within the group are arranged according to IBII. Frame I is a complete frame and does not require the filtering process. Frame B is a bidirectional reference frame and requires the filtering process to be performed so that the temporal filtering method is performed once every 4 frames. The image group supports random access to all GOPs whose length is a multiple of 4. Before the encoding step, a frame type determination is performed. If the current frame is the B frame, the filtered frame is obtained from the dynamic random access memory as the encoding input frame; otherwise, the original frame is obtained from the original image data.

5. The method according to claim 4, characterized in that, The method further includes: When the temporal filtering method is started, two invalid frames are counted first, so that the frame index of the frame processed by the temporal filtering method and the frame index of the encoding step are established with an offset of two frames. When the encoding step inserts an intraframe at frame index N, a complete frame is inserted at the corresponding position of the frame processed by the temporal filtering method. The corresponding position is determined according to the offset relationship to be the position of frame index N+2 of the frame processed by the temporal filtering method.

6. The method according to claim 1, characterized in that, The method further includes: At least one frame of shared space is reserved in the dynamic random access memory; After the time-domain filtering method completes filtering and stores the filtered frame in the shared space, the time-domain filtering method pulls up the completed filtering signal and pauses the filtering process. After receiving the completed filtering signal, the encoding step pulls the signal low and starts working. When the filtered frame is encoded, the filtered image encoding completion signal is pulled up and the work stops. After the time-domain filtering method reads the signal indicating that the filtered image encoding is complete, it lowers the signal and begins filtering the next frame.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the original pixels of the current frame to be filtered; The original pixels are first subjected to discrete cosine transform or discrete sine transform, and then quantization, inverse quantization and inverse transform are performed in sequence to obtain the reconstructed pixels; The rate distortion value is calculated based on the difference between the original pixel and the reconstructed pixel; The rate-distortion value is used to optimize the filtering intensity of the time-domain filtering method to determine the filtering intensity of the filtering process.

8. The method according to claim 1, characterized in that, The method further includes: When the rate-distortion optimization of the encoding step is completed and the next frame is a filtered frame, the input data sources of the pre-switching coarse motion estimation module, fine-tuned motion estimation module and fractional motion estimation module are switched to the motion compensation temporal filtering reference frame.

9. The method according to claim 1, characterized in that, The method further includes: The rate distortion value calculated by the rate distortion optimization module and the quantization parameters of the encoding steps are obtained; Based on the rate distortion value and the quantization parameter, bilateral filter parameters are calculated. The bilateral filter parameters include spatial domain filter parameters and tolerance domain filter parameters. The spatial domain filter parameters and the tolerance domain filter parameters are calculated using closed-form functions based on the rate distortion value and the quantization parameter.

10. The method according to claim 9, characterized in that, The method further includes: Calculate the mean and variance of the rate distortion values. The bilateral filter parameters are calculated based on the mean and variance of the rate-distortion values.

11. The method according to claim 9, characterized in that, The method further includes: The rate-distortion value and the quantization parameter are multiplied and divided, and the square root result is obtained by querying the pre-stored square root lookup table. The multiplication and division are 16-bit fixed-point operations, and the output of the bilateral filter parameter is a 10-bit fixed-point number.

12. The method according to claim 1, characterized in that, The method further includes: The frame type information is physically encoded in the address space of the dynamic random access memory; The dynamic random access memory includes a first storage area and a second storage area. The first storage area stores the original frame, and the second storage area stores the filtered frame. The data source for the encoding step is automatically routed by the high-order bits of the address. When the lower two bits of the current frame index are 0 or 1, the high-order bits of the address point to the second storage area; otherwise, they point to the first storage area.

13. The method according to claim 1, characterized in that, The method further includes: Configure N credit tokens; One credit token is consumed for each frame filtered. For each frame consumed during the encoding process, one credit token is returned. When the credit token count is 0, the time-domain filtering is paused; When the credit token count reaches N, the encoding step is paused; The number of credit tokens N=2, and the credit token counting bit width is 2 bits; The frame counter has a bit width of 16 bits.

14. The method according to claim 1, characterized in that, The method further includes: Obtain the quantization error energy and frequency domain distribution corresponding to the current frame to be filtered; The filtering intensity of the filtering process is determined based on the quantization error energy and frequency domain distribution. The filtering process is performed on the current frame to be filtered using the specified filtering strength. The quantization error energy bit width is 16 bits.

15. A hardware video encoder, characterized in that, include: The coarse motion estimation module, the fine-tuned optimized motion estimation module, and the fractional motion estimation module are used to perform downsampling and motion search for motion-compensated temporal filtering on the current frame to be filtered; Rate-distortion optimization module, used to perform interpolation and motion compensation required for motion-compensated temporal filtering on the current frame to be filtered; A filtering module is used to perform filtering processing on the current frame to be filtered to obtain a filtered frame; A dynamic random access memory is used to store the filtered frames as encoded input frames.

16. The hardware video encoder according to claim 15, characterized in that, The filtering module is also used for: Obtain the original pixels, interpolated optimal inter-frame predicted pixels, block partitioning results, and quantization parameters of the current coding block in the current frame to be filtered; Based on the original pixels, the interpolated optimal inter-frame predicted pixels, the block partitioning result, and the quantization parameters, a bilateral filter is used to perform filtering processing on the current frame to be filtered, resulting in filtered pixels. The filtered pixels are stored as the filtered frame in the dynamic random access memory.

17. The hardware video encoder according to claim 15, characterized in that, The dynamic random access memory includes a first storage area and a second storage area. The first storage area stores the original frame, and the second storage area stores the filtered frame. Frame type information is physically encoded in the address space of the dynamic random access memory; The data source for the encoding step is automatically routed by the high-order bits of the address. When the lower two bits of the current frame index are 0 or 1, the high-order bits of the address point to the second storage area; otherwise, they point to the first storage area.

18. The hardware video encoder according to claim 15, characterized in that, It also includes a credit token counting unit, which is used for: Configure N credit tokens; One credit token is consumed for each frame filtered. For each frame consumed during the encoding process, one credit token is returned. When the credit token count is 0, the time-domain filtering is paused; When the credit token count reaches N, the encoding step is paused; The number of credit tokens N=2, and the credit token counting bit width is 2 bits; The frame counter has a bit width of 16 bits.

19. A computer program product comprising a computer program that, when executed by a processor, implements the steps of a temporal filtering method in a preprocessing flow for a hardware video encoder as described in any one of claims 1 to 14.

Images