Video compression coder, decoder and system based on multi-level cache and parallel computing
By using multi-level caching and parallel computing in video compression encoders and decoders, the problems of high computational load and high storage space consumption in existing technologies are solved, achieving efficient video compression and decoding, and optimizing memory access and computation efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-03-27
AI Technical Summary
Existing video compression technologies involve high computational load and high storage space consumption during the encoding and decoding of block prediction modes, resulting in low efficiency and difficulty in meeting the requirements of high throughput and low latency. This problem is even more serious when multiple slices are processed in parallel.
We employ a video compression encoder and decoder based on multi-level caching and parallel computing. By dividing the image into multiple image blocks for parallel computation, and combining a multi-level caching system with a flatness detection search and pruning strategy, we optimize memory access and computation efficiency.
It significantly improves encoding speed and efficiency, reduces invalid BPV searches, optimizes memory bandwidth bottlenecks, reduces hardware design complexity and cost, and achieves efficient video compression and decoding.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention relates to the field of video compression technology, and in particular to a video compression encoder, decoder and system based on multi-level caching and parallel computing. Background Technology
[0002] The VESA Display Compression-M (VDC-M) standard is a display compression scheme for real-time low bit rate transmission. Its encoder supports multiple modes to adapt to different image content. Among them, the Block Prediction (BP) mode generates the optimal prediction vector (BPV) by searching the decoded reference pixels, thereby achieving a balance between distortion and bit rate.
[0003] On the encoding side, determining the optimal prediction vector (BPV) is computationally intensive, leading to inefficiency. The method provided by the VDC-M standard requires traversing a large number of candidate vectors over a wide search range and calculating the matching error for each candidate point. This directly results in extremely high computational costs and storage space consumption. In fact, all sub-blocks are searched indiscriminately within the same range, inevitably generating a large amount of unnecessary computation. Moreover, this frequent random access to a large number of reference pixels creates a memory bandwidth bottleneck, increasing the difficulty of encoder hardware design and making it difficult to meet the requirements of high throughput and low latency. On the decoding side, its "baseline" method is directly derived from the functional description of the VDC-M standard, resulting in excessive on-chip memory space consumption, thereby increasing chip area, power consumption, and cost. When multi-slice parallel processing is required, the above problems are exacerbated.
[0004] This shows that existing technologies cannot meet the encoding and decoding requirements of block prediction modes. Summary of the Invention
[0005] In view of this, it is necessary to provide a video compression encoder, decoder and system based on multi-level caching and parallel computing to solve the problem that existing technologies cannot meet the encoding and decoding requirements of block prediction mode.
[0006] To address the aforementioned problems, in a first aspect, the present invention provides a video compression encoder based on multi-level caching and parallel computing, comprising: The image segmentation module is used to divide the image to be compressed into multiple image blocks, and to divide the multiple image blocks into image prediction sub-blocks of a preset size; Multiple computing units, each including a first-level cache, are used to compute reference pixels of the image prediction sub-block in parallel, and the first-level cache of each computing unit is used to store the corresponding reference pixels; The secondary cache is used to store reconstructed pixels and provides cached data for each of the primary caches; A prediction vector determination module is used to configure an index for each of the computing units and update the reference pixels stored in the first-level cache based on the index to determine the optimal prediction vector for the image patch.
[0007] In one possible implementation, when a new image to be compressed needs to be compressed, the prediction vector determination module is used to update the index and update the reference pixels that need to be replaced in the primary cache from the secondary cache according to the new index.
[0008] In one possible implementation, the secondary cache uses a ring structure to cyclically store the reconstructed pixels.
[0009] In one possible implementation, the prediction vector determination module first determines the flatness information of the image prediction sub-block before determining the optimal prediction vector for the image prediction sub-block, and then determines the search strategy with the least computational load based on the flatness information.
[0010] In one possible implementation, when the flatness information of the image prediction sub-block is flat, a preset fast search strategy is adopted; when the flatness information of the image prediction sub-block is not flat, the search step size is adjusted.
[0011] Secondly, the present invention also provides a video compression decoder based on multi-level caching and parallel computing, comprising: A syntax parser for extracting at least one block prediction vector and quantization residual of a pixel block encoded by the encoder described in any of the above embodiments; A memory subsystem for storing the reconstructed pixels of the pixel block; A pipelined prediction unit is used to obtain reconstructed pixels from the memory subsystem based on the block prediction vector and generate prediction blocks; A reconstruction unit is used to combine the prediction block with a residual block derived from the quantized residual to generate a reconstruction block.
[0012] In one possible implementation, the memory subsystem includes a secondary cache consisting of multiple memory partitions and an intelligent memory controller, which is used to allocate memory access requests from the prediction unit and the display interface to different memory partitions.
[0013] In one possible implementation, the pipelined prediction unit includes: a first stage for calculating a predicted pixel address based on a block prediction vector; and a second stage for retrieving a predicted pixel from a memory subsystem using the predicted pixel address; wherein the operation of the first stage on the current block is performed concurrently with the operation of the second stage on the previous block.
[0014] In one possible implementation, the intelligent memory controller accesses multiple memory partitions in an interleaved manner.
[0015] Thirdly, the present invention also provides a video compression system based on multi-level caching and parallel computing, including the encoder and the decoder described in any of the above embodiments.
[0016] The beneficial effects of this invention are as follows: The video compression encoder based on multi-level caching and parallel computing provided by this invention adopts a highly parallel hardware architecture, including multiple parallel computing units. Each unit is responsible for searching reference pixels and calculating matching errors for image prediction sub-blocks. To solve the memory bandwidth bottleneck, this invention introduces a multi-level caching system, including a first-level cache located within each computing unit for storing frequently accessed reference pixels required for the current block search, and a shared, ring-structured second-level cache for cyclically storing reconstructed pixels and providing data updates to the first-level cache. By configuring indexes for the computing units and adopting a dynamic update strategy, this invention avoids repeated readings of the same pixel data and repeated address calculations. In addition, the encoder integrates a search pruning strategy based on flatness detection. By analyzing the texture complexity of image blocks, appropriate search strategies (such as adjusting the search step size or adopting a fast prediction mode) are dynamically selected for regions with different flatness, thereby effectively reducing invalid BPV searches and significantly improving encoding speed and efficiency. At the decoding end, a highly optimized memory subsystem and a deep pipelined processing architecture are adopted. This memory subsystem completely eliminates the large and redundant independent reconstruction buffers in traditional designs. This is achieved through the synergy of two key technologies: first, reconstructed block forward transmission, which transmits the latest decoded pixel block directly from the reconstruction unit output to the prediction unit input via a dedicated data path, bypassing memory access; and second, a unified, conflict-free block-level secondary buffer managed by an intelligent memory controller, providing virtual dual-port access for the display output stream and reference pixel read requests from the prediction unit, thereby eliminating memory access conflicts. The decoder also includes a decoupled multi-stage pipeline that separates BPV parsing, address generation, memory reads, and prediction calculations, enabling parallel processing of different data blocks, effectively hiding latency and maximizing hardware utilization. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 A video compression decoder architecture based on multi-level caching and parallel computing is provided for embodiments of the present invention; Figure 2 This is a schematic diagram of image segmentation provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an index update provided in an embodiment of the present invention; Figure 4 A flowchart illustrating a search strategy determination based on flatness is provided in an embodiment of the present invention; Figure 5 A VDC-M decoder structure diagram provided in an embodiment of the present invention; Figure 6 This invention provides an effective search range diagram for BP mode. Figure 7 This is a structural diagram of an intelligent storage controller module provided in an embodiment of the present invention; Figure 8 A row buffer scheduling graph provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a video compression system based on multi-level caching and parallel computing, provided for an embodiment of the present invention. Detailed Implementation
[0019] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.
[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.
[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0023] A specific embodiment of the present invention discloses a video compression encoder based on multi-level caching and parallel computing, comprising: The image segmentation module is used to divide the image to be compressed into multiple image blocks, and to divide the multiple image blocks into image prediction sub-blocks of a preset size; Multiple computing units, each including a first-level cache, are used to compute reference pixels of the image prediction sub-block in parallel, and the first-level cache of each computing unit is used to store the corresponding reference pixels; The secondary cache is used to store reconstructed pixels and provides cached data for each of the primary caches; A prediction vector determination module is used to configure an index for each of the computing units and update the reference pixels stored in the first-level cache based on the index to determine the optimal prediction vector for the image patch.
[0024] Furthermore, when a new image to be compressed needs to be compressed, the prediction vector determination module is used to update the index and update the reference pixels that need to be replaced in the primary cache from the secondary cache according to the new index.
[0025] Furthermore, the secondary cache uses a ring structure to cyclically store the reconstructed pixels.
[0026] In embodiments of the present invention, such as Figure 1 As shown, an encoder architecture is disclosed that aims to improve the encoding efficiency of BP mode through parallel processing, efficient cache management, and intelligent search strategies. The core of the encoder is a set of (e.g., 8) parallel computation units. At the start of encoding, an image partitioning module divides the image to be compressed into multiple 8x2 pixel image blocks, and further divides each image block into multiple smaller image prediction sub-blocks (e.g., 2x2 and 2x1 pixels). Subsequently, multiple computation units search for the best reference pixel in parallel within their respective search regions for these sub-blocks. To support this parallel computation and address memory bandwidth bottlenecks...
[0027] In this embodiment of the invention, each frame of the video to be encoded is taken as the image to be compressed and divided into standard-sized image blocks. Each image block is a fixed 8×2 pixels, composed of YCoCg or YCbCr components, and is the basic unit for candidate vector search and prediction matching in BP mode. Furthermore, each image block is further divided into multiple image prediction sub-blocks, and there are various ways to divide them.
[0028] Furthermore, for each image prediction sub-block, a computing unit can compute its reference pixels. The storage design of the reference pixels is optimized through multi-level caching. This includes a first-level cache, where the reference pixels of the current block are placed for BP mode candidate component search; a second-level cache, located in shared memory and configured as a circular cache, which updates the reference pixels required by the computing unit through the block index and stores the reconstructed block pixels; and a third-level cache, held in main memory, used for output pixel data.
[0029] Furthermore, a dynamic reference pixel update strategy is adopted, which binds the reference pixels in the first-level cache to an index to avoid repeatedly updating the same pixel data and recalculating the absolute address; when block processing advances to the next line, the second-level cache returns to the starting position to avoid switching between the upper and lower second-level caches; the use of the cache is optimized so that the storage of reference pixels is closely integrated with the block search process.
[0030] Furthermore, the encoder provided by this invention also incorporates a multi-level cache medium, including a primary cache and a secondary cache, with each computing unit having its own local primary cache. This cache stores reference pixels required for BPV searching of the current sub-block. Since the data is stored locally within the computing unit, parallel access speed is significantly improved, and the pressure on shared memory is reduced. All computing units share a single secondary cache, implemented in a circular structure to cyclically store recently reconstructed pixel blocks. This circular design avoids the complex operation of switching between secondary caches when processing new rows, simplifying the control logic.
[0031] Furthermore, to efficiently manage the data in the cache, this invention introduces an indexing mechanism. Each computational unit's first-level cache is bound to an index. When processing progresses to a new image patch, the prediction vector determination module only needs to update the index and update the reference pixel data that needs to be replaced in the first-level cache from the second-level cache according to the new index. This index-based dynamic update strategy avoids unnecessary reloading of unchanged reference pixels, further optimizing memory access efficiency.
[0032] In embodiments of the present invention, such as Figure 2As shown, the image patch is divided into multiple non-overlapping image prediction sub-blocks, each providing two partitioning schemes: a 2x2 partition and two upper and lower 2x1 partitions. A block prediction vector search operation is performed on each search region location within each partition of the block. For different partitioning schemes from the same image prediction sub-block, the 2x2 partition can be inferred from the results of the two 2x1 partitions of the same sub-block, thus reducing redundant searches and accelerating search efficiency. Since the two partitioning schemes overlap spatially, the search results of the 2×1 partition can be reused to reduce the redundant search overhead of the 2×2 partition.
[0033] Block prediction vector search specifically refers to the vector position search operation performed on the reconstructed reference region in VDC-M block prediction mode (BP mode) to find the optimal reference pixel location. Within a set of available reference pixel regions, an offset vector is searched and used to point to a block of reference data; this block is used as the prediction block. Then, the predicted value is added to the current block residual to achieve reconstruction. This offset vector used to indicate the reference block position is the block prediction vector (BPV). This operation of finding the block prediction vector is the block prediction vector search.
[0034] In this embodiment of the invention, for the same image prediction sub-block, multiple computing units can be used to search its reference pixels. Furthermore, the calculation formula for the local optimal prediction vector is: in, This is the local optimal prediction vector. The index value corresponding to the computing unit. Let be the internal offset of the calculation unit, where the internal offset of the calculation unit is determined by the sum of the minimum absolute differences of the calculation units, and the formula for calculating the sum of absolute differences is: Where SAD is the sum of absolute differences between computational units, and n is the number of image prediction sub-blocks. Predict the original pixels of the sub-block for the i-th image. The reference pixel for predicting the i-th sub-block of the image.
[0035] In this embodiment of the invention, in the BP (Block Prediction) mode of VDC-M, a coded block (or sub-partition) needs to select a prediction block in its reference region to replace the data of the original block, thereby reducing redundant information. From multiple candidate prediction blocks, the one most similar to the current block is selected. This "similarity" is measured by "distortion"; the smaller the distortion, the more accurate the prediction. For each coded sub-partition (e.g., 2×1): each BPV points to a reference pixel block; the error between this reference block and the current original block is calculated to obtain the "distortion" value; the BPV with the smallest distortion value is selected; this BPV is used as the search result for that sub-partition.
[0036] In embodiments of the present invention, such as Figure 3 As shown, the index update process is as follows: for indices 1 to 3, the indexes are updated to 0 to 2; for index 0, the index is updated to 3. For indices 5 to 7, the indexes are updated to 4 to 6; and for index 4, the index is updated to 7. After updating the indexes, the reference pixel data update strategy is executed. For index 0 in the first-level cache, the upper half of the reconstructed pixel data (1×8 data) and the last pixel data (a total of 9 pixels) are updated to the area at index 1. The calculation units for indices 1, 2, 4, 5, and 6 remain unchanged. For index 3, the reference pixel data for calculation units at indices 0 and 1 are updated by nine consecutive pixels at the current block index + 3 in the second-level cache. For the data at index 7, the upper half of the nine pixels (the last column of pixels from the previous reconstructed block plus the pixel data from the current reconstructed block) is updated to index 0, and the lower half is updated to index 1. Finally, the lower half of the reconstructed block data is updated to the position pointed to by the current index in the second-level cache, and the current block index + 1 is updated to the new current block index.
[0037] Furthermore, the first step is to update the index of the search area. The update steps are as follows: In the upper 2x1 partition search area, the original indexes 1 to 3 are updated to 0 to 2, and the original index 0 is updated to 3; in the upper 2x1 partition, the original indexes 5 to 7 are updated to 4 to 6, and the original index 4 is updated to 7.
[0038] In the search area of the lower 2x1 partition, the original index 0 remains unchanged, the original indexes 2 to 3 are updated to 1 to 2; the original index 1 is updated to 3; in the search area of the lower 2x1 partition, the original indexes 5 to 7 are updated to 4 to 6, and the original index 4 is updated to 7.
[0039] The second step is to update the pixel data in the search area and the secondary cache. The update steps are as follows: Once the current block's reconstructed pixel data is obtained, for updating the pixel data in the search area, the upper 2x1 partition search area index 3 writes data from the block with the current block index + 3 in the secondary cache to update, and the upper 2x1 partition search area index 7 writes data from the first row of the current block's reconstructed pixel data to update.
[0040] The next 2x1 partition searches region index 0 by reconstructing pixel data from the first row of the current block and writing it. Index 3 updates the data by writing data from the block at index + 3 of the current block in the secondary cache; index 7 updates the data by reconstructing pixel data from the second row of the current block and writing it.
[0041] In some possible embodiments of the present invention, before determining the optimal prediction vector for the image prediction sub-block, the prediction vector determination module first determines the flatness information of the image prediction sub-block, and determines the search strategy with the least search computation based on the flatness information.
[0042] In embodiments of the present invention, such as Figure 4 As shown, to reduce unnecessary computational overhead, a flatness detection-based search pruning strategy is integrated into the encoder. Before performing a BPV search on a predicted sub-block of an image, the encoder first determines the flatness information of the sub-block. Flatness detection determines the texture complexity by analyzing the changes in pixel values within the sub-block (e.g., calculating the sum of the absolute values of AC coefficients through Hadamard transform) and classifies it into types such as "very flat", "slightly flat", "flat to complex transition", or "complex".
[0043] Specifically, in image compression, "flatness determination" is not a single metric, but rather a series of principled strategies for identifying regions in an image that are "efficiently compressible." The core idea is to prioritize and process regions with gradual changes and low information entropy more efficiently in exchange for a higher compression ratio. A region is considered "flat" not merely because it has a single color, but more accurately because its pixel values change very slowly in the spatial or transform domain, possessing very low high-frequency energy. By utilizing the low information content of flat regions, they are represented with as few bits as possible, thus allocating the saved bits to complex, detail-rich regions, achieving globally optimal visual quality at a given total bitrate.
[0044] Furthermore, when the flatness information of the image prediction sub-block is flat, a preset fast search strategy is adopted; when the flatness information of the image prediction sub-block is not flat, the search step size is adjusted.
[0045] In this embodiment of the invention, based on the determined flatness information, the encoder selects a corresponding search strategy for the sub-block. For blocks detected as "very flat," the conventional full-range BPV search can be skipped, and a fast selection module can be directly entered. This module compares the SAD values of only a few predefined prediction vectors (such as the pixel directly above or the most recently reconstructed pixel column) and selects the optimal one as the result, thereby greatly accelerating the processing speed. For other types of blocks, different search step sizes can be allocated according to their complexity, avoiding a pixel-by-pixel full search operation on the entire search area, thereby improving search efficiency while ensuring quality.
[0046] In this embodiment of the invention, taking one example, is_flat_block=1 indicates that the detection result is a flat block, and the conventional BPV search module can be skipped to enter the fast selection module. The search strategy of the fast selection module is that the prediction vector can be divided into two types: the first type is copied from the eight consecutive pixels above the current block position to the corresponding position; the second type is from the nearest column of pixel values of the most recently reconstructed pixel. The SAD of these two types is compared and the one with the smallest value is selected as the search result. Flatness detection is based on the changes in pixel values in the current block to determine its texture complexity, thereby classifying it into different flatness types (FlatnessType), which is usually used to optimize the selection of prediction vectors. The pixel values of the current image prediction sub-block are used as input, and the RGB input is first transformed into the YCoCg color space. For each component, an 8×2 or 4×2 / 4×1 Hadamard+Haar transformation is performed. After excluding the DC coefficients, the absolute values of all AC coefficients are calculated and normalized and weighted to obtain complexityCur; if the input bit depth bpc≠8, a right shift and scaling are also performed. Whether the previous block is complex (isPrevBlockComplex) is determined by comparing complexityPrev, complexityCur, and the maximum inline complexity (maxLineComplexity). Whether the next block is flat (isNextBlockFlat) is determined by thresholding complexityNext, complexityCur, and maxLineComplexity. A boolean array of length 3, isFlatToComplex[], is maintained to capture the transition from flat to complex and advances with each block clock cycle. Flatness is categorized as follows: 0: VeryFlat; 1: SomewhatFlat; 2: Complex to Flat transition; 3: Flat to Complex transition. Block search operations execute different search strategies based on the flatness information provided by the search region. Flatness types include flat, complex to flat, flat to complex, complex, and other types. The search strategy assigns a search step size to each search partition type to avoid a full search of the search region, further accelerating search efficiency.
[0047] Specifically, the flatness determination relies entirely on the "complexity" value. The encoder quantizes the texture intensity of the pixel domain into a numerical value through frequency domain transformation. For an 8×2 block, a two-stage transformation is performed: row transformation: an 8-point Hadamard Transform is applied to each row of pixels. For chroma components of 4:2:0 / 4:2:2, a 4-point transformation is used. Column transformation: a 2-point Haar Transform (i.e., simple column sum / difference) is applied to each column result. Normalization: the transformed coefficients are shifted left or right according to the bit depth (bpc) (hadTotalShift) to ensure the values are within a uniform dynamic range. The sum of the absolute values of all AC coefficients is calculated. Note that DC components (i.e., coefficients at coordinate (0,0)) must be excluded. If the input bit depth > 8 bpc, the calculated sum needs to be right-shifted by bpc - 8 bits to normalize it to 8-bit equivalent precision. The final value obtained is the complexity of the block.
[0048] Before final classification, two Boolean logic states need to be calculated: isPrevBlockComplex (whether the previous block is complex) and isNextBlockFlat (whether the next block is flat).
[0049] Determine the complexity of the previous block: For the first block of a line, it is true if Prev > 90 and Cur ≤ 50; for the last block of a line, it is true if Cur ≤ 50. For the middle block (4:4:4 / 4:2:2), first calculate the dynamic threshold, threshold = (maxLineComplexity >> 1) + (maxLineComplexity >> 3) (approximately 0.625 times the maximum complexity), and it is true if any of the following conditions are met: Condition 1: Next≤6 (the next block is extremely flat); Condition 2: Prev>threshold and Next≤25 (the previous block is more complex than the maximum value of the current line, and the next block is relatively flat).
[0050] For the middle block (4:2:0), it is true if Next≤20 and Prev>40.
[0051] Determine the complexity of the next block: If the current block is an edge column (first or last column), it is false.
[0052] Otherwise, both of the following conditions must be met simultaneously: 1. Large decrease: (complexityCur-complexityNext)>(maxLineComplexity>>4).
[0053] 2. The absolute value of the next block is low: complexityNext < ((maxLineComplexity>>2) + (maxLineComplexity>>3)).
[0054] Final flatness type classification: The encoder checks conditions in the following priority order; if the conditions are met, the lock type is determined. If none of the conditions are met, it is marked as a "non-flat block". Type 0: Very Flat Condition: complexityCur≤1.
[0055] Action: QP is significantly reduced (even to negative) for handling solid colors or extremely smooth gradients.
[0056] Type 1: Somewhat Flat Condition 1 <complexityCur≤3。
[0057] Action: Reduce QP appropriately Type 2: Complex-to-Flat Transition Conditions: isPrevBlockComplex (true) and isNextBlockFlat (true).
[0058] Action: Force a reduction in QP.
[0059] Significance: When a texture is about to end (e.g., transitioning from a leaf texture to a sky background), the QP must be lowered in advance to prevent ringing or noise "overflow" into flat areas caused by the previous high QP.
[0060] Type 3: Flat-to-Complex Transition State tracking: A block-time delay is introduced using the shift register isFlatToComplex array.
[0061] Mutation detection conditions: The previous area is very flat: complexityPrev≤3.
[0062] The next block is very complex: complexityNext ≥ 50.
[0063] Stability check: No such mutation has occurred in the past two time points.
[0064] Final determination: Whether the current block is Type3 depends on the state calculated at the previous block time (i.e., isFlatToComplex[1]). If a block satisfies multiple conditions at the same time (e.g., it satisfies both Type0 and Type2 logic), the type with the smaller index has higher priority. For example: if flatType0 is true and flatType2 is also true, then the final determination is Type0. The embodiments of the present invention employ an intelligent pruning strategy, enabling the encoder to concentrate computing resources on complex regions that truly require fine-grained searching, thereby significantly reducing the overall computational load and power consumption.
[0065] The encoder for compressing video streams provided in this invention employs a highly parallel hardware architecture. The core of this architecture consists of multiple parallel computing units, each responsible for searching reference pixels and calculating matching errors for image prediction sub-blocks. To address memory bandwidth bottlenecks, this invention introduces a multi-level caching system, including a first-level cache within each computing unit for storing frequently accessed reference pixels required for the current block search, and a shared, ring-structured second-level cache for cyclically storing reconstructed pixels and providing data updates to the first-level cache. By configuring indexes for the computing units and employing a dynamic update strategy, this invention avoids repeated readings of the same pixel data and repeated address calculations. Furthermore, the encoder integrates a flatness detection-based search pruning strategy. By analyzing the texture complexity of image blocks, it dynamically selects appropriate search strategies (such as adjusting the search step size or adopting a fast prediction mode) for regions with different flatness levels, thereby effectively reducing invalid BPV searches and significantly improving encoding speed and efficiency.
[0066] On the other hand, the present invention also provides a decoder for compressing video streams, comprising: A syntax parser for extracting at least one block prediction vector and quantization residual of a pixel block encoded by the encoder described in any of the embodiments; A memory subsystem for storing the reconstructed pixels of the pixel block; A pipelined prediction unit is used to obtain reconstructed pixels from the memory subsystem based on the block prediction vector and generate prediction blocks; A reconstruction unit is used to combine the prediction block with a residual block derived from the quantized residual to generate a reconstruction block.
[0067] Furthermore, the memory subsystem includes a secondary cache consisting of multiple memory partitions and an intelligent memory controller, which is used to allocate memory access requests from the prediction unit and the display interface to different memory partitions.
[0068] Furthermore, the pipelined prediction unit includes: a first stage for calculating the predicted pixel address based on the block prediction vector; and a second stage for obtaining the predicted pixel from the memory subsystem using the predicted pixel address; wherein the operation of the first stage on the current block and the operation of the second stage on the previous block are performed concurrently.
[0069] Furthermore, the intelligent memory controller accesses multiple memory partitions in an interleaved manner.
[0070] In this embodiment of the invention, the decoder solves the memory redundancy and performance bottleneck problems of the prior art through innovative memory subsystem and pipeline design. The core innovation of the decoder lies in its optimized memory subsystem, which completely eliminates the need for an independent reconstruction buffer and significantly reduces the hardware area.
[0071] In this embodiment of the invention, the optimized memory subsystem includes a reduced secondary cache array. Unlike existing technologies that require three secondary caches, this invention employs a half-line latency mechanism, requiring only two physical secondary caches to operate, reducing secondary cache memory requirements by 33.3%. The physical SRAM of each secondary cache is logically divided into at least two partitions (e.g., an "even array" and an "odd array"). The controller is responsible for scheduling memory access requests from the display interface and the pipelined prediction and reconstruction units. By scheduling these requests to different groups of partitions at different clock cycles, the controller utilizes a single-port SRAM to implement a virtual dual-port function, ensuring conflict-free concurrent execution of both types of access, thereby eliminating pipeline stalls. A dedicated, register-based data path routes the newly reconstructed 8x2 pixel block directly from the output of the reconstruction unit back to the input of the prediction unit. This allows the data to be immediately used to predict the next block, completely bypassing the memory subsystem, and is one of the key technologies for eliminating independent reconstruction buffers. Only a small number of prediction reference pixels that cannot be obtained through the secondary cache or forward pass path are used to store this data.
[0072] Furthermore, to fully utilize the non-blocking memory subsystem, the decoder's prediction and reconstruction logic is constructed as a deep, multi-stage pipeline. It receives the BPV and calculates the memory addresses of all predicted pixels. Based on the generated addresses, the intelligent memory controller retrieves predicted pixel data from the grouped L2 buffer, compact prediction buffer, and forward propagation path. The retrieved pixels are used to generate a prediction block, and simultaneously, the quantization residual is dequantized. The prediction block is then added to the residual block to generate the final reconstructed block. This reconstructed block is simultaneously written to the L2 buffer and sent to the forward propagation path. This deep pipeline structure enables time-parallel processing; for example, while stage 4 is reconstructing block N, stages 3, 2, and 1 can process blocks N+1, N+2, and N+3, respectively. This effectively hides the latency of each operation, ensuring that the decoder can continuously output reconstructed blocks at a stable, high rate, achieving the architecture's theoretical maximum throughput.
[0073] In this embodiment of the invention, the intelligent memory module is not a single memory, but a subsystem composed of a controller and a specific storage architecture, including an intelligent memory controller, a line cache, and other data storage structures. The intelligent memory controller is the core control unit, responsible for managing all memory access requests. Its main function is to allocate the required pixel data for a periodic distribution from different sources (prediction unit and display interface). Through logical partitioning and intelligent scheduling, the intelligent memory module achieves high-performance parallel data read and write using only a single-port SRAM and less physical memory capacity, effectively supporting real-time decoding processing of 4K UHD.
[0074] Furthermore, such as Figure 5 The diagram shows the VDC-M decoder architecture, which uses two FIFOs to separate the DFE from the DBE and the DBE from the display. This is because the flow rates between each module are inconsistent; specifically for the DFE, the required decoding cycles may vary each time.
[0075] At the same time, the FIFO can also function as a pipeline register. For the display-side FIFO, a ping-pong buffer structure is used, that is, two FIFOs, which are switched each time a line of data is read.
[0076] like Figure 6 As shown, the effective search range for BP mode is A0~A7, B0~B24, and C0~C65. The blank cells represent the positions of the currently required encoding or decoding blocks. Figure 7The diagram shows the structure of the intelligent storage controller module. The reconstruction block and reconstruction buffer store registers containing data types, while the line buffer is an SRAM structure. The line buffer stores pixel data for the row below the reconstruction block. Line buffer BANK 0 stores even-numbered block index pixels, and BANK 1 stores odd-numbered block index pixels. The intelligent storage controller is a core component of the VDC-M decoder back-end (DBE) memory subsystem. It solves the storage redundancy and access conflict problems in high-resolution video compression hardware design by using complex timing scheduling and physical memory partitioning. Based on the BP's search range and the data stored in the reconstruction block and reconstruction buffer within the intelligent storage controller module, the intelligent controller needs to control the reading of the line buffer so that other pixels within the search range can be read. Figure 8 The diagram shows the line buffer scheduling, using a four-cycle pipeline as an example. This architecture reduces the number of line buffers to 2 lines, with a latency of half a line. Specifically, if the total number of blocks in a line is not odd, the latency ends at the beginning of the preceding odd-numbered block in the middle position.
[0077] The decoder provided in this invention employs a highly optimized memory subsystem and a deeply pipelined processing architecture. This memory subsystem completely eliminates the large and redundant independent reconstruction buffer found in traditional designs. This is achieved through the synergy of two key technologies: first, forward transmission of reconstruction blocks, which transmits the latest decoded 8x2 pixel block directly from the reconstruction unit output to the prediction unit input via a dedicated data path, bypassing memory access; second, a unified, conflict-free grouped secondary buffer, managed by an intelligent memory controller, providing virtual dual-port access for the display output stream and reference pixel read requests from the prediction unit, thereby eliminating memory access conflicts. The decoder also includes a decoupled multi-stage pipeline that separates BPV resolution, address generation, memory read, and prediction calculation stages, enabling parallel processing of different data blocks, effectively hiding latency and maximizing hardware utilization.
[0078] In some possible embodiments of the present invention, such as Figure 9 As shown, a video compression system based on multi-level caching and parallel computing is also provided, including the encoder and the decoder described in any of the foregoing embodiments.
[0079] In this embodiment of the invention, both the encoder and decoder can utilize hardware resource sharing. For example, the adder in the decoder used for BP mode reconstruction can be shared with the reconstruction path of the transform decoding mode. The input is selected according to the current encoding mode by a multiplexer, thereby reducing the total number of gates. Significant system-level advantages are achieved through collaborative optimization of the encoder and decoder. The efficient parallel processing and intelligent search strategy at the encoding end, combined with the streamlined memory architecture and non-blocking pipeline at the decoding end, constitute a high-performance, low-cost VDC-M encoding and decoding solution. This solution achieves a gate count saving of over 30% at the decoder backend, while supporting 4K UHD real-time processing performance of 4 pixels per cycle, paving the way for the deployment of advanced display compression technologies in area- and power-constrained applications.
[0080] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A video compression encoder based on multi-level caching and parallel computing, characterized in that, include: The image segmentation module is used to divide the image to be compressed into multiple image blocks, and to divide the multiple image blocks into image prediction sub-blocks of a preset size; Multiple computing units, each including a first-level cache, are used to compute reference pixels of the image prediction sub-block in parallel, and the first-level cache of each computing unit is used to store the corresponding reference pixels; The secondary cache is used to store reconstructed pixels and provides cached data for each of the primary caches; A prediction vector determination module is used to configure an index for each of the computing units and update the reference pixels stored in the first-level cache based on the index to determine the optimal prediction vector for the image patch.
2. The video compression encoder based on multi-level caching and parallel computing according to claim 1, characterized in that, When a new image to be compressed needs to be compressed, the prediction vector determination module is used to update the index and update the reference pixels that need to be replaced in the primary cache from the secondary cache according to the new index.
3. The video compression encoder based on multi-level caching and parallel computing according to claim 1, characterized in that, The secondary cache uses a ring structure to cyclically store the reconstructed pixels.
4. The video compression encoder based on multi-level caching and parallel computing according to claim 1, characterized in that, Before determining the optimal prediction vector for the image prediction sub-block, the prediction vector determination module first determines the flatness information of the image prediction sub-block, and then determines the search strategy with the least computational load based on the flatness information.
5. The video compression encoder based on multi-level caching and parallel computing according to claim 4, characterized in that, When the flatness information of the image prediction sub-block is flat, a preset fast search strategy is adopted; when the flatness information of the image prediction sub-block is not flat, the search step size is adjusted.
6. A video compression decoder based on multi-level caching and parallel computing, characterized in that, include: A syntax parser is used to extract at least one block prediction vector and quantization residual of a pixel block encoded by the encoder according to any one of claims 1 to 5; A memory subsystem for storing the reconstructed pixels of the pixel block; A pipelined prediction unit is used to obtain reconstructed pixels from the memory subsystem based on the block prediction vector and generate prediction blocks; A reconstruction unit is used to combine the prediction block with a residual block derived from the quantized residual to generate a reconstruction block.
7. The video compression decoder based on multi-level caching and parallel computing according to claim 6, characterized in that, The memory subsystem includes a secondary cache consisting of multiple memory partitions and an intelligent memory controller. The intelligent memory controller is used to allocate memory access requests from the prediction unit and the display interface to different memory partitions to achieve conflict-free parallel access.
8. The video compression decoder based on multi-level caching and parallel computing according to claim 7, characterized in that, The pipelined prediction unit includes: a first stage for calculating the predicted pixel address based on the block prediction vector; and a second stage for retrieving the predicted pixel from the memory subsystem using the predicted pixel address; wherein the operation of the first stage on the current block is performed concurrently with the operation of the second stage on the previous block.
9. The video compression decoder based on multi-level caching and parallel computing according to claim 7, characterized in that, The intelligent memory controller accesses multiple memory partitions in an interleaved manner.
10. A video compression system based on multi-level caching and parallel computing, comprising the encoder as described in any one of claims 1 to 5 and the decoder as described in any one of claims 6 to 9.