A method for caching inter prediction reference frame data based on sram

Abstract

The application discloses an interframe prediction reference frame data caching method based on SRAM, relates to the technical field of digital video coding, and is suitable for H.266 / VVC standard. The method realizes two-dimensional storage and management of reference frame data by constructing a ring-shaped storage structure formed by multiple double-port SRAMs in an alternating stacking manner. Two-dimensional addressing mapping is adopted, the storage position is calculated according to the CTU coordinate, the preloading and cyclic coverage updating of data are supported, read-write parallel operation is realized by using the double-port SRAM, multiple rows of pixel data are read at a time by using the alternating stacking storage, and the reference block across the CTU boundary is quickly read out in the motion estimation search window range to realize matching and complete motion estimation. The application improves the reuse rate and access efficiency of the reference frame data, reduces the bandwidth dependence and access delay of the off-chip memory, effectively accelerates the motion estimation process in interframe prediction, and improves the overall performance of the video encoder.

Description

Technical Field

[0001] This invention relates to the field of digital video coding technology, and in particular to an SRAM-based method for caching inter-frame prediction reference frame data. Background Technology

[0002] Video coding technology aims to achieve efficient data compression by eliminating spatial, temporal, and statistical redundancy in video data. Inter-frame prediction is a core technology in video coding that utilizes temporal redundancy. It uses motion estimation to find the best matching block for the current coding unit in an already encoded reference frame. The motion vector between the current block and the best matching reference block is the optimal motion vector. By encoding only the optimal motion vector and retaining the residual data, compression efficiency is significantly improved.

[0003] In the next-generation video coding standard H.266 / VVC, in pursuit of higher compression efficiency, larger coding unit units (CTUs, e.g., a maximum of 128x128 pixels) and more flexible coding unit (CU) partitioning methods have been introduced. This has led to a corresponding expansion of the search range for motion estimation, a significant increase in the amount of data that needs to be accessed from the reference frame, and more complex access patterns, posing serious challenges to the bandwidth and latency of the storage system.

[0004] In existing hardware encoder implementations, reference frame data is typically stored in large-capacity but slow-access external dynamic random access memory (such as DDR or DRAM). DRAM usually uses linear addresses to store two-dimensional image data, i.e., storing each row of pixels consecutively. However, motion estimation in inter-frame prediction requires accessing consecutive or non-consecutive data blocks (such as CUs) in two-dimensional space. When reading a two-dimensional data block that spans the storage row boundaries, multiple discrete DRAM access requests need to be initiated. This access mode is not only inefficient, but also suffers from unpredictable data readiness time due to high DRAM access latency and potential interference from requests from other modules in the system, severely limiting the real-time processing performance of video encoding.

[0005] To alleviate the aforementioned problems, existing technologies have proposed using on-chip static random access memory (SRAM) to temporarily store data from the currently processed region in the reference frame. However, the simple SRAM caching strategy still has significant shortcomings. If only the search window data of a single CU is cached, the entire cache content needs to be refreshed when switching to the next CU after processing one CU, introducing a large amount of data transfer overhead. At the same time, without optimizing the data storage structure inside the SRAM, reading a two-dimensional data block may still require multiple access operations, failing to fully utilize the high-speed characteristics of SRAM. Therefore, how to design an efficient on-chip caching architecture and management method that matches the access characteristics of H.266 / VVC inter-frame prediction data to maximize data reuse, minimize access latency, and support flexible and efficient two-dimensional data block reading has become a key issue in improving the performance of video encoding hardware. Summary of the Invention

[0006] The purpose of this invention is to provide an SRAM-based inter-frame prediction reference frame data caching method to solve the problems mentioned in the background art.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] An SRAM-based inter-frame prediction reference frame data buffering method includes:

[0009] A cache space is constructed by organizing multiple dual-port SRAMs in an alternating stacking manner, and the cache space is configured as a ring storage structure;

[0010] The ring memory structure is managed using a two-dimensional addressing mapping method, including:

[0011] Before processing the current CTU, the reference frame data containing the motion estimation search window of the current CTU is preloaded into the cache space; while processing the current CTU, the reference frame data of the next CTU is preloaded in parallel.

[0012] After processing the current CTU, the motion estimation search window slides horizontally on the reference frame and iteratively processes the CTUs in the current reference frame according to the coverage update principle.

[0013] As a preferred embodiment of the present invention, the dual-port SRAM is used to perform parallel read and write operations, including: one port for reading reference frame data required for the current CTU motion estimation calculation, and the other port for writing reference frame data preloaded for the CTU.

[0014] As a preferred embodiment of the present invention, the storage method of the ring-shaped storage structure is as follows:

[0015] The storage logic of the cache space is mapped to a two-dimensional plane with 4 columns and 3 rows of CTUs. The target logical column is obtained by taking the modulo of the column coordinates of the current CTU with 4. The target logical column The value corresponds to the reference storage address of the current CTU in the two-dimensional plane;

[0016] The reference block of the CU pointed to by the motion vector is obtained by superimposing the reference storage address of the current CTU with the motion vector, and the CU reference block is compared with the pixel data of the current CU to complete the motion estimation calculation.

[0017] If the current CTU's row coordinate switches to the next row, the read and write states of the ring storage structure are reset.

[0018] As a preferred embodiment of the present invention, the two-dimensional plane is configured to include a storage space of 12 CTUs, wherein the storage space of 9 CTUs is used for caching reference frame data in the motion estimation search window, and the storage space of 3 CTUs is used for preloading reference frame data.

[0019] As a preferred embodiment of the present invention, the alternating stacking method includes:

[0020] If it is possible to read out p rows of pixel data at once, then 2p parallel dual-port SRAMs are constructed;

[0021] When a data request is issued for address m of 2p parallel dual-port SRAMs, the data blocks corresponding to address m of the 2p SRAMs are read simultaneously and formed into a CU reference block with a maximum of p rows.

[0022] As a preferred embodiment of the present invention, the method of managing the writing of the ring memory structure using two-dimensional addressing mapping includes:

[0023] Based on the two-dimensional coordinates (CTU_X, CTU_Y) of the CTU, calculate the one-dimensional logical address Addr. This one-dimensional logical address Addr is used to preload the CTU to be preloaded from external memory to the corresponding reference memory address in the two-dimensional plane. The expression is:

[0024] ;

[0025] ;

[0026] In the formula, N is the total number of columns in the two-dimensional planar logic partition. The predefined base storage address for the corresponding target logical column, For the column coordinates of CTU, The row offset is determined based on the row coordinate CTU_Y. The target logical column is determined based on the column coordinates.

[0027] As a preferred embodiment of the present invention, the method of managing the reading of the ring memory structure using two-dimensional addressing mapping includes:

[0028] The target logical column is calculated based on the column coordinates of the current CTU. According to the target logical column The horizontal displacement mv_x of the motion vector is determined by a preset rule to obtain a reference storage address, and the vertical displacement mv_y is added to the reference storage address to obtain the read data address; the read data address is modulo k to obtain the read address of the SRAM; where k is the k rows of pixel data read simultaneously in the alternating stacking mode; the preset rule includes: if the horizontal displacement mv_x is negative, the value of the target logical column OFFSET_X is reduced by 1, and if the horizontal displacement mv_x is positive or zero, the value of the target logical column OFFSET_X remains unchanged.

[0029] As a preferred embodiment of the present invention, when the CU read by the motion estimation calculation crosses two adjacent CTU boundaries in the reference frame, a data block covering the width of the two adjacent CTUs is read at once. The read data block is reassembled and spliced ​​according to its position in the two-dimensional plane in the reference frame to form a temporary data matrix containing the target reference block and adjacent regions. According to the position of the target reference block pointed to by the current motion vector in the reference frame, the region corresponding to the target reference block in the temporary data matrix is ​​determined, and the target reference block data corresponding to the current CU size is extracted from the region.

[0030] As a preferred embodiment of the present invention, the coverage update principle includes: if there is a CTU in the current cache space that has the same target logical column as the CTU that newly enters the motion estimation search window, and the CTU has already performed motion estimation calculation, then the CTU that newly enters the motion estimation search window will cover the corresponding CTU.

[0031] As a preferred embodiment of the present invention, the motion estimation search window covers a 3-row × 3-column CTU area, wherein the size of the CTU is 32×32 pixels, 64×64 pixels or 128×128 pixels.

[0032] The beneficial effects of this invention are as follows: By introducing a ring storage structure, the CTUs within the motion estimation search window of the reference frame are preloaded and reused cyclically, avoiding frequent access to external memory (such as DRAM) by each CU. Combined with the alternating stacking method and two-dimensional addressing mapping, multiple rows of pixel data can be read in parallel at one time, significantly reducing the number of data requests and access latency, and effectively improving the data reading speed during motion estimation calculation. Using dual-port SRAM as the storage medium, it supports parallel operation of writing data to be preloaded on one port and reading the current search data on the other port, eliminating read-write conflicts, improving cache throughput and overall processing efficiency, and is especially suitable for the real-time requirements of high-resolution video encoding. By employing a two-dimensional addressing mapping and alternating data stacking method, not only can CU data of arbitrary location and size be quickly located and read, but also complete data can be obtained through a single read and truncation operation when the reference block crosses the boundary of adjacent CTUs. This adapts to the flexible CU partitioning method in H.266 / VVC, enhancing applicability and efficiency. By caching and reusing reference frame data in on-chip SRAM, the frequency and amount of access to external DRAM are significantly reduced, lowering storage bandwidth requirements and facilitating high-performance video coding on resource-constrained hardware platforms. The combined effect of these technologies makes the data supply for motion estimation calculations in inter-frame prediction more efficient and stable, shortening coding time and improving the real-time performance and energy efficiency of video compression processing, making it particularly suitable for the high-complexity coding environment of the next-generation video coding standard H.266 / VVC. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0034] Figure 1 This is a flowchart of an SRAM-based inter-frame prediction reference frame data caching method according to the present invention.

[0035] Figure 2 This is a two-dimensional plan view in an embodiment of the present invention;

[0036] Figure 3 This is a conversion diagram of the ring storage structure in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of a dual-port SRAM in an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of alternating stacked SRAMs in an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of the storage mapping of data in alternately stacked SRAMs in an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram illustrating the address correspondence of alternately stacked SRAMs in an embodiment of the present invention.

[0041] Figure 8 This is a schematic diagram showing the CU located between two CTUs in an embodiment of the present invention;

[0042] Figure 9 This is a flowchart of motion estimation calculation in an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0044] like Figures 1-8 As shown, this is an embodiment of the present invention, which provides an SRAM-based inter-frame prediction reference frame data caching method, including:

[0045] A cache space is constructed by organizing multiple dual-port SRAMs in an alternating stacking manner, and the cache space is configured as a ring storage structure;

[0046] The storage method of ring memory structure is as follows Figures 2-3As shown, before performing motion estimation calculations on CTU(1,1), nine CTUs—CTU(0,0), CTU(1,0), CTU(2,0), CTU(0,1), CTU(1,1), CTU(2,1), CTU(0,2), CTU(1,2), and CTU(2,2)—are preloaded. After preloading, motion estimation calculations are performed on CTU(1,1). The motion vector is obtained by searching around the top-left corner of CTU(1,1) as the origin. These nine CTUs constitute a motion estimation search window. The search range is set to ±64, and the CTU size is 64×64 pixels. At the same time, the CTUs (3,0), (3,1), and (3,2) required for the motion estimation calculation of the next CTU (2,1) are preloaded. After the motion estimation calculation of CTU (1,1) is completed and the CTUs (3,0), (3,1), and (3,2) required for the motion estimation calculation of CTU (2,1) are preloaded, the motion estimation search window moves one CTU to the right and the motion estimation calculation of CTU (2,1) begins.

[0047] When preloading CTU(4,0), CTU(4,1), and CTU(4,2), CTU(0,0), CTU(0,1), and CTU(0,2) are overwritten based on the circular storage structure, and motion estimation calculation is performed on CTU(3,1). The operation is repeated until all motion estimation calculations for the current CTU row are completed, and then the system is reset to perform the preloading data and motion estimation calculation for the next row. The purpose of resetting is to prevent the column coordinates from being multiples of 4.

[0048] The aforementioned storage space can only store one reference frame. If the current frame is a P-frame or a B-frame, multiple reference frames are required. For example, N reference frames require N 768×64-byte storage spaces. Each reference frame independently stores its corresponding data. Multi-reference frame image prediction technology is used to improve prediction accuracy, allowing one image to reference multiple frames. For example, CTU(0,0) in the 4th frame references the 3rd image as a reference frame, and CTU(1,0) references the 2nd image as a reference frame. According to the technical protocol, a maximum of 15 frames can be referenced forward and backward. This invention only requires the two or three images closest to the current reference frame, so two 768×64-byte spaces are used to store the two forward images. If the current frame is a B-frame, reference frames are needed in both forward and backward directions, so an additional two 768×64-byte spaces are used to store the two backward images, and four storage spaces are used to store the reference frames. If more than two images are needed, they are read directly from external DDR.

[0049] like Figure 4As shown, dual-port SRAM is divided into two types: pseudo dual-port and true dual-port. Both ports of a true dual-port have complete read and write functions, while port A of a pseudo dual-port is used for writing and port B is used for reading.

[0050] like Figure 5 As shown, to achieve the simultaneous reading of 4 rows of pixel data, eight dual-port SRAMs are created, labeled a, b, c, d, e, f, g, and h. Each SRAM has a bit width of 64 bytes, a depth of 96, and a memory size of 768 × 64 bytes. Figure 6 As shown, a-0 represents the first row of SRAM a, b-0 represents the first row of SRAM b, c-0 represents the first row of SRAM c, and so on; a-1 represents the second row of SRAM a, b-1 represents the second row of SRAM b, c-1 represents the second row of SRAM c, and so on; until the last row of each SRAM from a-95 to h-95 is completed; each row stores one row of pixel data. If a data request is issued to address 0 of the eight SRAMs, four rows of 128 bytes of data will be read, and each byte represents one pixel data. Figure 7 As shown, the memory cells at addresses 0 to 15 of SRAM a, b, c, and d constitute a CTU memory space.

[0051] In the alternating stacking mode, when a data request is issued to eight parallel SRAM addresses 0, eight 64-byte data blocks (a-0, b-0, c-0, d-0, e-0, f-0, g-0, and h-0) ​​are read simultaneously. Each address can read data from the eight SRAMs and form a 4-row 128-byte CU (i.e., four rows of CUs, each with 128 pixels) for motion estimation calculation. If a 64×64 pixel CTU needs to be read, it is done by reading 16 addresses.

[0052] like Figure 8 As shown, if a CU (Cubic Array) pixel is 64×64 and this CU is located between two reference frames' CTUs, then the two 64-byte data points a-0 and e-0 are read and combined into a 128-byte data block. This block is then truncated to obtain a 64-byte data block spanning both CTUs. Therefore, the data width read in a single row is set to 128 bytes. Since the 64×64 pixel CU spans two CTUs, addresses 16 to 31 of SRAM (a, b, c, d, e, f, g, and h) are read simultaneously, and a 64×128-byte data block is read. This block is then truncated to obtain a 64×64 pixel CU for motion estimation calculation.

[0053] The ring memory structure is managed using a two-dimensional addressing mapping method for reading and writing.

[0054] In the reference frame, the CTU generates corresponding two-dimensional number information such as (CTU_X, CTU_Y), where CTU_X is the column coordinate and CTU_Y is the row coordinate. The target logical column OFFSET_X is obtained by taking the modulo of 4 using the column coordinate. If the value of the target logical column OFFSET_X is 0, 1, 2, or 3, the corresponding reference storage addresses in the two-dimensional plane are 64, 256, 448, and 640, respectively. The CTU is written with the upper left corner of the current CTU as the origin and the row offset OFFSET_Y. When a new row coordinate is encountered, a reset operation is performed.

[0055] For example, load the first CTU(0,1) with row coordinate 1 (CTU_X=0, CTU_Y=1) and initialize the target logical column OFFSET_X to 0; since the row offset OFFSET_Y of CTU(0,0) compared to CTU(0,1) is negative 1, read CTU(0,0) and place it in SRAM at addresses 0-15 in slots a, b, c, and d, corresponding to base memory addresses 0-63; since the row offset OFFSET_Y of CTU(0,1) is 0, read CT... U(0,1) is placed in SRAM at addresses 16-31 in slots a, b, c, and d, corresponding to reference storage addresses 64-127. Since the row offset OFFSET_Y of CTU(0,2) compared to CTU(0,1) is 1, CTU(0,2) is read and placed in SRAM at addresses 32-47 in slots a, b, c, and d, corresponding to reference storage addresses 128-191. The first three columns and three rows of CTU are filled in sequentially according to the above, and the motion estimation calculation of CTU(1,1) begins.

[0056] When performing motion estimation calculation on CTU(1,1), CTU(3,0), CTU(3,1), and CTU(3,2) in the 4th column are written to the cache space in advance, and the motion estimation calculation of CU is only performed in the area of ​​the motion estimation search window. After the motion estimation calculation of CTU(1,1) is completed, since the CTU direction is vertical, the next motion estimation search window is shifted horizontally to the right by one CTU distance. Therefore, the column of CTU(0,0), CTU(0,1), and CTU(0,2) is freed up and replaced by CTU(4,0), CTU(4,1), and CTU(4,2) for overwriting, thus completing a set of circular storage.

[0057] If the range of the motion estimation search window is set to ±64, then the search range of the motion vector (mv_x, mv_y) is also ±64, corresponding to -64 to 63. If the horizontal displacement mv_x is negative, the target logical column OFFSET_X is reduced by 1 to obtain the reference storage address. If the horizontal displacement mv_x is positive or zero, the reference storage address of the target logical column OFFSET_X remains unchanged. The vertical displacement mv_y is added to the reference storage address to obtain the read data address. Since the alternating stacked storage method can read 4 rows of pixel data in one parallel operation, taking the read data address modulo 4 gives the corresponding SRAM read address.

[0058] like Figure 2 As shown, if the current CTU column coordinate for motion estimation is 5 and the horizontal displacement mv_x of the motion vector is -24 and the vertical displacement mv_y is -20, then the target logical column OFFSET_X is 1, and the corresponding reference storage address is 256. Since the horizontal displacement mv_x is negative, the reference storage address corresponding to the target logical column OFFSET_X minus 1 is 64. Therefore, the starting point of the current CU is the reference storage address 64. The vertical displacement mv_y is -20, which is added to the reference storage address 64, resulting in 44. Since the current CU's pixels are 16×16, the reference storage addresses to be read are 44 to 59. After alternating stacking, a maximum of 4 rows of pixel data are read each time. Therefore, the data at addresses 11, 12, 13, and 14 of SRAM a, b, c, and d are read out, thus completing a 16-row reference frame CU data with each row being 16 bytes.

[0059] like Figure 9 As shown, this is the second embodiment of the present invention, which provides inter-frame prediction when the current frame is a P-frame, including:

[0060] S1. The video signal is input into the video encoding terminal in the encoding order of each frame. According to the encoding sequence, the current frame is divided into I-frame, P-frame or B-frame. I-frame is used for intra-frame prediction, while P-frame and B-frame are used for inter-frame prediction. Since the current frame input is a P-frame, the motion estimation search window data of the first reference frame of 2 frames needs to be pre-loaded. The range of the motion estimation search window is set to ±64, and the CTU is 64×64 pixels.

[0061] S2. Since the row coordinate of the first CTU is 0, CTU(0,0), CTU(1,0), CTU(0,1), and CTU(1,1) are loaded from the external DDR to the four CTUs at the base memory addresses 64-127, 128-191, 256-219, and 320-383 respectively to complete the preloading. The P-frame loads the CTU(0,0), CTU(1,0), CTU(0,1), and CTU(1,1) of the two images in the forward direction.

[0062] S3. Start motion estimation calculation for CTU(0,0). Perform motion vector search algorithm, optimal motion vector prediction and matching algorithm, and sub-pixel interpolation operation based on integer pixel inference in CTU(0,0), CTU(1,0), CTU(0,1), and CTU(1,1) to complete the motion estimation calculation of the current CTU. For example, the horizontal displacement mv_x of the motion vector predicted by an 8×16 pixel CU is 8, and the vertical displacement mv_y is 10. Since the column coordinate is 0, the reference storage address for reading data is 64. Because the vertical displacement mv_y is 10, the reference storage address 64 is offset by 10 in the direction of the vertical displacement mv_y, resulting in a reference storage address of 74. Taking the modulo of the reference storage address 74 divided by 4 gives 18 remainder 2, so the starting addresses for reading SRAM are 18, 19, and 20, resulting in 12 rows and 128 bytes of data. Based on the 8×16 pixel CU, rows 3 to 10 (8 rows in total) are extracted, and for each row, data from Byte 8 to Byte 23 (16 bytes in total) is extracted. Reading the reference block of this 8×16 pixel CU requires 3 address requests to the SRAM. If the request were made to the DRAM, since there are 8 rows of addresses, 8 address requests would be needed to the DRAM. For a 64×64 pixel CTU, there are 32 8×16 pixel CUs. Therefore, based on the SRAM-based inter-frame prediction reference frame data caching method of this invention, reading one CU reference block for the current CU can save 160 read data requests. Furthermore, each CU needs to read multiple CU reference blocks during motion estimation, resulting in a significant overall performance improvement. In motion estimation calculation, each CU reads multiple candidate CU reference blocks to complete the matching through a search algorithm (such as hexagonal search). Since the on-chip SRAM cache used in this invention has extremely low fixed access latency (data is usually returned in the next clock cycle after the request is issued) and supports overlapping read and write operations with parallel ports through alternating stacking, it can greatly reduce the data readiness waiting time and eliminate problems such as queuing delay, bus contention, and uncontrollable access time caused by accessing off-chip DRAM. In summary, this solution significantly improves the efficiency of motion estimation calculation and overall coding by reducing the number of requests, hiding data loading latency, and providing deterministic high-speed data supply.

[0063] S4. While performing motion estimation calculation on CTU(0,0), preload the reference frame data of CTU(2,0) and CTU(2,1). After completing the above preloading and motion estimation calculation operations, further perform motion estimation calculation on CTU(1,0) and simultaneously preload the data of CTU(3,0) and CTU(3,1).

[0064] S5. When it is necessary to preload the data of CTU(4,0) and CTU(4,1), the column coordinate CTU_X is 4, and taking the modulo of 4 gives the target logical column OFFSET_X as 0. Therefore, the data of CTU(4,0) and CTU(4,1) will be written to the storage area corresponding to column 0 (base storage address 64-127), overwriting the oldest and no longer needed data of CTU(0,0) and CTU(0,1) already stored in that column; this process realizes the first overwrite update of the ring storage structure (i.e., ring switching).

[0065] S6. Similarly, when a complete row of CTUs is completed, a reset is performed when a new row of CTUs is started, and the data is reloaded.

[0066] S7. Inter-frame prediction for one frame is completed when all CTUs are completed.

[0067] like Figure 9 As shown, this is the second embodiment of the present invention, which provides inter-frame prediction when the current frame is a B-frame, including:

[0068] S1. The video signal is input into the video encoding terminal in the encoding order of each frame. According to the encoding sequence, the current frame is divided into I-frame, P-frame or B-frame. I-frame is used for intra-frame prediction, while P and B frames need to be used for inter-frame prediction. Since the current frame input is a B-frame, the motion estimation search window data of the two preceding reference frames needs to be preloaded, and the reference frame data of the two following frames needs to be loaded. The range of the motion estimation search window is set to ±64, and the CTU is 64×64 pixels.

[0069] S2. Since the row coordinate of the first CTU is 0, CTU(0,0), CTU(1,0), CTU(0,1), and CTU(1,1) are loaded from the external DDR to the four CTUs at reference storage addresses 64-127, 128-191, 256-219, and 320-383 in the two-dimensional plane to complete the preloading; since the current frame input is a B frame, CTU(0,0), CTU(1,0), CTU(0,1), and CTU(1,1) of the two forward images and CTU(0,0), CTU(1,0), CTU(0,1), and CTU(1,1) of the two backward images are loaded.

[0070] S3. Perform motion estimation calculation on CTU(0,0) to obtain the reference frame index of CTU(0,0), which is the first forward frame and the first backward frame (because the B frame is obtained by completing the forward and backward frames to obtain two motion vectors and then performing weighted synthesis). In the frames corresponding to the indexes, perform motion vector search algorithm, best motion vector prediction and matching algorithm on CTU(0,0), CTU(1,0), CTU(0,1), and CTU(1,1), as well as sub-pixel interpolation operation based on integer pixels to complete the motion estimation calculation of the current CTU.

[0071] S4. While performing motion estimation calculation on CTU(0,0), preload the reference frame data of CTU(2,0) and CTU(2,1) for the two forward and two backward frames. After loading and motion estimation calculation are completed, perform motion estimation calculation on CTU(1,0) while continuing to preload CTU(3,0) and CTU(3,1).

[0072] S5. When starting motion estimation calculation for CTU(1,0), since the reference frame index of CTU(1,0) is in the 7th forward frame and the 6th backward frame, and is not in the preloaded reference frame, a search for CTU(1,0) in the reference frame is requested from the external DDR to complete the motion estimation calculation of the current CTU.

[0073] S6. While performing motion estimation calculation on CTU(1,0), preload CTU(3,0) and CTU(3,1) of the reference frames before and after it. After loading and motion estimation calculation are completed, perform motion estimation calculation on CTU(1,0) and continue to preload the data of the two frames before and after CTU(3,0) and CTU(3,1).

[0074] S7. And so on. When the entire row of CTUs is completed, a new row of CTUs is started, and the data is reset and reloaded.

[0075] S8. Inter-frame prediction for one frame is completed when all CTUs are completed.

[0076] In summary, the ring storage structure adopted in this invention can pre-cache and continuously maintain a reference frame data area larger than a single motion estimation search window (e.g., containing 12 CTUs). When processing spatially adjacent CTUs in a video sequence, the data involved in the sliding of the motion estimation search window highly overlaps in the cache. Through the overwrite update principle, newly loaded CTUs only replace old CTUs that are no longer needed, enabling efficient recycling of data in the cache. The multiple data requests that originally needed to be initiated to external DRAM for each CU or even for each search are centrally optimized into predictive batch preloading on a CTU-by-CTU basis. The direct effect is that it greatly reduces the frequency of accessing high-latency external memory, reduces system bus congestion and waiting time, and thus provides a continuous and high-speed data supply for motion estimation calculation.

[0077] By employing multiple dual-port SRAMs organized in an interleaving manner, this invention achieves wide-bit-width, parallel data reading. For example, a single address request can simultaneously read four rows of pixel data from eight SRAMs, forming a complete data block. This design fundamentally changes the traditional linear storage model that requires reading line by line, minimizing the number of access cycles required to read a two-dimensional data block. The read and write ports of the dual-port SRAMs are independent, supporting fully parallel execution of data preloading (writing) and motion estimation (reading), achieving overlap of "computation-data handling," hiding data loading latency, and significantly improving the overall throughput of the cache and the utilization of processing units.

[0078] This invention's innovative two-dimensional addressing mapping mechanism directly and efficiently maps the two-dimensional coordinates (CTU_X, CTU_Y) and motion vectors (mv_x, mv_y) of the CTU to the one-dimensional physical address of the ring memory structure. It supports fast location and reading of CUs of any position and size in H.266 / VVC without the need for complex address translation overhead. For the common case where the CU reference block spans two CTU boundaries, this method solves the problem of non-aligned access by reading the cross-boundary wide data block at once and then truncating it, thus minimizing the number of accesses, simplifying data path design, and ensuring processing efficiency.

[0079] This invention shifts data access from off-chip DRAM with uncontrollable latency to on-chip SRAM with deterministic and extremely low latency, making the data readiness time for motion estimation calculations deterministic and predictable. This is crucial for ensuring the pipeline stability of real-time encoding of high-definition and ultra-high-definition video, and improves the real-time performance and reliability of the entire encoder system.

[0080] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any other combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product, which includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0081] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0082] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for buffering inter-frame prediction reference frame data based on SRAM, characterized in that, include: A cache space is constructed by organizing multiple dual-port SRAMs in an alternating stacking manner, and the cache space is configured as a ring storage structure; The alternating stacking method includes: If it is possible to read p rows of pixel data at once, then 2p parallel dual-port SRAMs are constructed; when a data request is issued to address m of the 2p parallel dual-port SRAMs, the data blocks corresponding to address m of the 2p SRAMs are read at the same time and formed into a CU reference block with a maximum of p rows. The storage method of the ring-shaped storage structure is as follows: The storage logic of the cache space is mapped to a two-dimensional plane with 4 columns and 3 rows of CTUs. The target logical column is obtained by taking the modulo of the column coordinates of the current CTU with 4. The target logical column The value corresponds to the reference storage address of the current CTU in the two-dimensional plane; the reference storage address of the current CTU is superimposed with the motion vector to obtain the CU reference block pointed to by the motion vector, and the CU reference block is compared with the pixel data of the current CU to complete the motion estimation calculation; if the row coordinate of the current CTU switches to the next row, the read and write states of the ring storage structure are reset; the read and write of the ring storage structure are managed by two-dimensional addressing mapping, including: The method of managing the writing of the ring memory structure using two-dimensional addressing mapping includes: Based on the two-dimensional coordinates (CTU_X, CTU_Y) of the CTU, calculate the one-dimensional logical address Addr. This one-dimensional logical address Addr is used to preload the CTU to be preloaded from external memory to the corresponding reference memory address in the two-dimensional plane. The expression is: Y In the formula, N is the total number of columns in the two-dimensional planar logic partition. The predefined base storage address for the corresponding target logical column, For the column coordinates of CTU, Y is the row offset determined based on the row coordinate CTU_Y. The target logical column is determined based on the column coordinates; The method of managing the reading of the ring memory structure using two-dimensional addressing mapping includes: The target logical column is calculated based on the column coordinates of the current CTU. According to the target logical column The horizontal displacement mv_x of the motion vector is determined by a preset rule to obtain a reference storage address, and the reference storage address is added to the vertical displacement mv_y to obtain the read data address; the read data address is modulo k to obtain the SRAM read address; where k is the k rows of pixel data read simultaneously in the alternating stacking mode; the preset rule includes: if the horizontal displacement mv_x is negative, the value of the target logical column OFFSET_X is reduced by 1; if the horizontal displacement mv_x is positive or zero, the value of the target logical column OFFSET_X remains unchanged; Before processing the current CTU, the reference frame data containing the motion estimation search window of the current CTU is preloaded into the cache space; while processing the current CTU, the reference frame data of the next CTU is preloaded in parallel. After processing the current CTU, the motion estimation search window slides horizontally on the reference frame and iteratively processes the CTUs in the current reference frame according to the coverage update principle. The coverage update principle includes: if there is a CTU in the current cache space that has the same target logical column as the CTU that newly enters the motion estimation search window, and the CTU has already performed motion estimation calculation, then the CTU that newly enters the motion estimation search window will cover the corresponding CTU.

2. The SRAM-based inter-frame prediction reference frame data buffering method according to claim 1, characterized in that, The dual-port SRAM is used to perform parallel read and write operations, including: one port for reading reference frame data required for the current CTU motion estimation calculation, and the other port for writing reference frame data preloaded for the CTU.

3. The SRAM-based inter-frame prediction reference frame data buffering method according to claim 2, characterized in that, The two-dimensional plane is configured to include storage space for 12 CTUs, of which 9 CTUs are used for caching reference frame data for motion estimation search windows, and 3 CTUs are used for preloading reference frame data.

4. The SRAM-based inter-frame prediction reference frame data buffering method according to claim 3, characterized in that, When the CU read by the motion estimation calculation crosses two adjacent CTU boundaries in the reference frame, a data block covering the width of the two adjacent CTUs is read at once. The read data block is reassembled and spliced ​​according to its position in the two-dimensional plane in the reference frame to form a temporary data matrix containing the target reference block and adjacent regions. Based on the position of the target reference block pointed to by the current motion vector in the reference frame, the region corresponding to the target reference block in the temporary data matrix is ​​determined, and the target reference block data corresponding to the current CU size is extracted from this region.

5. The SRAM-based inter-frame prediction reference frame data buffering method according to claim 1, characterized in that, The motion estimation search window covers a 3-row × 3-column CTU area, where the size of the CTU is 32×32 pixels, 64×64 pixels, or 128×128 pixels.

Images