Bitmap mask screening cache method for sparse matrix operation
By using a bitmap mask to filter cache data, redundant data in sparse matrix operations is eliminated in real time, solving the problems of wasted cache resources and low memory access bandwidth utilization, and improving computing performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2026-05-08
- Publication Date
- 2026-06-05
AI Technical Summary
Existing block caching mechanisms suffer from wasted cache resources and low memory access bandwidth utilization when handling sparse matrix operations, resulting in redundant and invalid data occupying hardware storage resources and reducing computing performance.
A bitmap mask filtering and caching method is adopted. The bitmap mask is generated through preprocessing and processed by a three-level hardware pipeline at the hardware acceleration end to remove redundant data in real time. The effective data is compactly stored by using an address compression mapping strategy.
This improved cache payload utilization, increased bus memory access bandwidth utilization, reduced on-chip storage footprint, and ensured that computing performance was not affected by memory access latency.
Smart Images
Figure CN122152767A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer architecture and data processing technology, and specifically relates to a bitmap mask filtering and caching method for sparse matrix operations. Background Technology
[0002] In cutting-edge computing fields such as scientific computing, deep learning, graph neural networks, and finite element analysis, sparse matrix operations are one of the core computational mechanisms. These matrices have extremely high sparsity, with a typically very low percentage of non-zero elements.
[0003] To save storage space, the industry typically uses compressed formats (such as CSR, COO, etc.) to store sparse matrices, that is, storing only the values of non-zero elements and their corresponding row and column indices. However, this storage and computation model brings a very special memory access characteristic: a lack of spatial locality. During execution... In typical operations such as multiplying a sparse matrix by a vector, the access address of the source vector x depends entirely on the column index of the non-zero elements in the sparse matrix. Since non-zero elements are usually distributed as a large number of isolated points, the memory access of the source vector by the computing unit is characterized by highly random and discontinuous indirect addressing.
[0004] The core concept of on-chip cache design in modern computer architecture and hardware accelerators is based on the "locality principle" of data access. To mitigate the high access latency of main memory, general-purpose caches typically employ a block-based caching mechanism, loading data in fixed cache lines. When the processor requests data at a specific address, the cache controller prefetches the contiguous physical memory block containing that data as a whole into the on-chip cache. However, this classic hardware design logic contradicts the discrete data distribution characteristics of sparse matrices. In large-scale sparse computation scenarios, using a traditional block-based cache architecture (e.g., employing a 64B cache block to store 16 32-bit single-precision floating-point numbers) will lead to the following problems:
[0005] Because the required effective data points are highly discrete, a single load of consecutive data blocks typically contains only a very small number of effective elements for computation, with the effective payload often accounting for less than 10%. This results in a large amount of unused redundant zero data or irrelevant data being moved and stored in the cache, which not only occupies hardware storage resources but also pushes out originally useful data from the cache, reducing the utilization rate of cached data.
[0006] In large-scale computing tasks with high concurrency across multiple cores, the frequent transmission of redundant and invalid information on the bus causes a large amount of data with no computational value to occupy the on-chip / off-chip memory access bandwidth. The truly needed valid data cannot be delivered to the computing unit in a timely manner due to bandwidth congestion, thus limiting the performance of the entire system due to the "memory wall" and reducing the system's computing power. Summary of the Invention
[0007] This invention provides a bitmap mask filtering and caching method for sparse matrix operations to solve the aforementioned technical problems, specifically adopting the following technical solution:
[0008] A bitmap mask filtering and caching method for sparse matrix operations includes the following steps:
[0009] Preprocessing stage: The host generates a bitmap mask based on the spatial distribution characteristics of the non-zero elements of the sparse matrix and pre-downloads it to the local storage of the hardware acceleration end;
[0010] During runtime: The hardware acceleration module performs real-time validity identification and rejection of the input vector data stream through a three-stage hardware pipeline, which includes:
[0011] Data packet parsing and matching: Parse the data packet header information and match the corresponding bitmap mask;
[0012] Data splitting: Perform bit-by-bit matching based on the bitmap mask to filter valid data;
[0013] Effective data caching: An address compression mapping strategy is used to compactly write effective data into the on-chip target cache.
[0014] Furthermore, in the preprocessing stage, the host scans the column indices of the non-zero elements of the sparse matrix, divides the source vector into several vector data blocks according to the data volume of a single bus data packet, marks whether the elements in each data block are referenced by non-zero elements, generates a bitmap mask table containing vector data block indices and data validity marks, and pre-distributes the bitmap mask table to the local on-chip storage unit of each computing core on the hardware acceleration side.
[0015] Furthermore, the bitmap mask is stored in a segmented form, with each segment corresponding to the filtering information of a vector data block. The vector data block index contains the segmentation information of the bitmap mask, and the data validity mark contains the corresponding bitmap mask. 1 and 0 are used to distinguish between vector data that will participate in the operation when multiplying a sparse matrix into a vector and vector data that will not participate in the operation.
[0016] Furthermore, during the bitmap mask generation process, if all elements in a certain vector data block are not referenced by the non-zero elements of the sparse matrix, the filtering information corresponding to that data block is directly discarded and its corresponding vector data block index is not retained, so as to reduce the storage overhead of the bitmap mask table.
[0017] Furthermore, the runtime phase is independently managed by the hardware acceleration end. The three-level hardware pipeline splits the data receiving, filtering and diversion, effective extraction, and cache writing operations into a pipeline level according to clock cycles, so that the entire process delay of the data packet from input to effective writing is controlled within three clock cycles.
[0018] Furthermore, in the data packet information parsing and matching step, when the processing core receives the source vector data packet, it simultaneously completes the reception of the data packet and the parsing of the packet header, extracts the data source identifier and data length, uses the parsed data source identifier as a hardware address index to match with the vector data block index, and retrieves the corresponding bitmap mask from the local information table.
[0019] Furthermore, in the data splitting step, the bitmap mask is read from the local library according to the extracted index, the corresponding data payload is temporarily stored in the register, and the received vector data stream is matched bit by bit with the bitmap mask according to the preset filtering granularity. This makes the data units with a validity mark of 1 in the bitmap mask recognized as valid data and retained, while the data units with a validity mark of 0 are recognized as invalid redundant data and discarded directly.
[0020] Furthermore, in the effective data caching step, the effective data stream after the previous cleaning and splitting is received and a write operation is triggered. The address mapping logic no longer follows the original discrete address distribution of the sparse matrix, but generates continuous auto-incrementing addresses. The address compression mapping strategy is used to compactly push the effective elements into the cache line of the target cache.
[0021] Furthermore, in the effective data caching step, effective data of different types is segmented and stored in different cache blocks to ensure the alignment of the data physical addresses and provide hardware-level address alignment support for the parallel reading of multiple computing cores in the future.
[0022] Furthermore, the hardware acceleration end is an FPGA accelerator, with an on-chip data bus width set to 1024 bits. A single bus transmission can carry 32 32-bit single-precision floating-point numbers. The bitmap mask sequence has a bit width of 32 bits, corresponding to the number of data units transmitted in a single bus transmission.
[0023] The advantage of this invention lies in the bitmap mask filtering caching method for sparse matrix operations, which overcomes the problems of wasted cache resources and low memory access bandwidth utilization in existing block caching mechanisms when processing sparse matrices. By performing hardware-level real-time filtering before data is written to the cache, redundant and invalid data is eliminated, thereby improving the actual effective cache load utilization. Without introducing additional memory access latency, it significantly reduces on-chip storage footprint and increases effective bandwidth throughput.
[0024] This invention eliminates a large amount of redundant and invalid data in real time at the physical layer by constructing a three-stage hardware pipeline at the front end of the data write cache link, including "information parsing and matching, data diversion, and valid data caching". This mechanism not only eliminates cache pollution caused by sparse matrix indirect addressing, stabilizing the cache effective load ratio to 100%, but also effectively improves the bus memory access bandwidth utilization. At the same time, combined with an address compression mapping strategy for valid data, this invention can effectively reduce the total occupancy of the on-chip "source vector" cache, freeing up valuable on-chip storage resources in hardware accelerators such as FPGAs. In addition, thanks to the architecture design of the hardware pipeline, the entire filtering process can be executed quickly, ensuring that while obtaining storage and bandwidth optimization benefits, no additional memory access latency overhead is introduced to the overall system computation. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is a schematic diagram illustrating the principle of bitmap mask generation;
[0027] Figure 2 This is a schematic diagram of the bitmap mask table structure;
[0028] Figure 3 This is a schematic diagram of a three-level hardware pipeline architecture. Detailed Implementation
[0029] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0030] This application discloses a bitmap mask filtering and caching method for sparse matrix operations, comprising the following steps:
[0031] Preprocessing stage: The host generates a bitmap mask based on the spatial distribution characteristics of the non-zero elements of the sparse matrix and pre-downloads it to the local storage of the hardware acceleration end.
[0032] Runtime phase: The hardware acceleration stage uses a three-stage hardware pipeline to perform real-time validity identification and elimination of the input vector data stream. The three-stage hardware pipeline includes:
[0033] Data packet parsing and matching: Parse the data packet header information and match the corresponding bitmap mask.
[0034] Data splitting: Perform bit-by-bit matching based on the bitmap mask to filter valid data.
[0035] Effective data caching: An address compression mapping strategy is used to compactly write effective data into the on-chip target cache.
[0036] Specifically, the bitmap mask filtering and caching method for sparse matrix operations in this application adopts a collaborative processing architecture of "software preprocessing + hardware pipeline acceleration", which includes two time-series steps: the preprocessing stage and the runtime stage.
[0037] In the preprocessing stage, the host computer scans the spatial distribution characteristics of non-zero elements in the sparse matrix to pre-generate bitmap masks for each computation core. These masks use binary bits to indicate whether each element in the vector data block will be referenced in subsequent sparse matrix-vector multiplication operations. The mask table is then sent to the local storage of the hardware acceleration unit. During runtime, the hardware acceleration unit independently performs real-time filtering through a three-stage hardware pipeline: the first stage, "packet information parsing and matching," parses the packet header information and matches the corresponding bitmap mask. The second stage, "data splitting," performs bit-by-bit logical matching on the data payload based on the mask, retaining valid data and eliminating invalid and redundant data. The third stage, "valid data caching," uses an address compression mapping strategy to generate continuously incrementing addresses and compactly writes the filtered valid data into the on-chip target cache. This entire method achieves efficient coordination between preprocessing and runtime through hardware-software decoupling, low-latency real-time filtering through hardware pipelines, and eliminates the fragmentation problem of sparse data storage through address compression mapping.
[0038] This invention constructs a three-stage hardware pipeline at the front end of the data write cache link, eliminating redundant and invalid data in real time at the physical layer. This ensures that the cache effective load ratio remains stable at 100%, eliminating the cache pollution problem caused by sparse matrix indirect addressing. Combined with an address compression mapping strategy, it effectively reduces the total occupancy of the on-chip source vector cache, freeing up valuable on-chip storage resources in hardware accelerators such as FPGAs. Thanks to the pipeline architecture design, the latency of the entire filtering process is controlled within three clock cycles, achieving storage and bandwidth optimization benefits without introducing additional memory access latency overhead to the overall system computation.
[0039] like Figure 1 and Figure 2As shown in the embodiments of this application, in the preprocessing stage, the host scans the column indices of the non-zero elements of the sparse matrix, divides the source vector into several vector data blocks according to the data volume of a single bus data packet, marks whether the elements in each data block are referenced by non-zero elements, generates a bitmap mask table containing vector data block indices and data validity marks, and pre-distributes the bitmap mask table to the local on-chip storage unit of each computing core on the hardware acceleration side.
[0040] Specifically, the host side uses the column indices of the non-zero elements of the sparse matrix as a basis to divide the source vector into several equal-length vector data blocks according to the data packet capacity of a single transmission on the hardware bus. For each element within a data block, it checks whether it is referenced by a non-zero element of the sparse matrix, thereby generating a bitmap mask table containing vector data block indices and data validity markers. This mask table is pre-distributed to the dedicated on-chip storage units of each processing core on the hardware acceleration side, providing data preparation for fast hardware retrieval during runtime. This preprocessing strategy transforms the address dependencies that originally needed to be dynamically calculated at runtime into a statically searchable mask table, thus shifting the computational load from runtime to the preprocessing stage.
[0041] By moving the generation and distribution of bitmap masks to the preprocessing stage, the runtime overhead of dynamic parsing of sparse matrix structures is avoided. This allows the hardware acceleration end to quickly obtain filtering information by performing table lookup matching at runtime, significantly reducing runtime computational complexity and latency. At the same time, dividing vector data blocks according to the granularity of bus data packets aligns the mask granularity with the hardware transmission unit, facilitating subsequent pipelined parallel bit-by-bit matching operations.
[0042] In the embodiments of this application, the bitmap mask is stored in a segmented form, with each segment corresponding to the filtering information of a vector data block. The vector data block index contains the segmentation information of the bitmap mask, and the data validity mark contains the corresponding bitmap mask. 1 and 0 are used to distinguish between vector data that will participate in the operation when multiplying a sparse matrix into a vector and vector data that will not participate in the operation.
[0043] Specifically, the bitmap mask employs a segmented storage structure, with each segment uniquely corresponding to the filtering information of a vector data block. The vector data block index field carries the position identifier of that segment within the overall mask table, while the data validity flag field, in binary bitmap form, carries the validity status of each element within the data block. A 1 indicates that the vector data at the corresponding position will be referenced in the sparse matrix-vector multiplication operation, while a 0 indicates that it will not be referenced. This data structure compresses the complex address mapping relationship into a compact bitmap form, requiring only one binary bit for each element to express its validity status, significantly reducing the storage volume of the filtering information. Furthermore, the bit-by-bit structure of the bitmap naturally adapts to parallel hardware matching operations.
[0044] The segmented storage structure of the bitmap mask makes the filtering information of each data block independently addressable, which makes it easy for the hardware to quickly locate the target mask segment through simple index calculation. The use of binary encoding of 1 and 0 reduces the mask storage overhead to a theoretical minimum and is naturally compatible with the AND gate operation of hardware logic circuits. It can complete the parallel filtering and judgment of multiple bits of data in a single clock cycle, thereby improving the filtering throughput.
[0045] In the preprocessing stage, this invention generates a dedicated filtering information table for each computational core based on the spatial distribution characteristics of non-zero elements and a multi-core task partitioning strategy. The filtering information uses '1' and '0' to distinguish between vector data that will participate in the sparse matrix-vector multiplication operation and those that will not. Figure 1 As shown. This information table is stored in segments in the form of bitmap masks, with each segment corresponding to the filtering information of a vector data block. Its core fields include "vector data block index" and "data validity marker". The "vector data block index" contains the segmentation information of the bitmap mask, while the "data validity marker" contains the corresponding bitmap mask, such as... Figure 2 As shown.
[0046] In the embodiments of this application, the runtime phase is independently taken over by the hardware acceleration end. The three-level hardware pipeline splits the data receiving, filtering and diversion, effective extraction, and buffer writing operations into pipeline stages according to clock cycles, so that the entire process delay of the data packet from input to effective writing is controlled within three clock cycles.
[0047] Specifically, the runtime phase is entirely managed autonomously by the hardware acceleration unit, requiring no host intervention, ensuring decoupling of the filtering process from the main computation flow. The three-tier hardware pipeline strictly decomposes the four operations—data reception, filtering and routing, valid data extraction, and cache writing—according to clock cycles, allowing adjacent data packets to be processed simultaneously at different stages of the pipeline, creating a pipeline parallel effect. Through meticulous timing design, the entire process latency of a single data packet from entering the pipeline to completing the valid data cache writing is strictly constrained to within three clock cycles. This latency constraint ensures that the filtering mechanism will not become a bottleneck on the system's memory access path.
[0048] In the embodiments of this application, in the data packet information parsing and matching step, when the processing core receives the source vector data packet, it simultaneously completes the reception of the data packet and the parsing of the packet header, extracts the data source identifier and data length, uses the parsed data source identifier as a hardware address index to match with the vector data block index, and retrieves the corresponding bitmap mask from the local information table.
[0049] Specifically, when the processing core receives a source vector data packet from the on-chip bus, this step synchronously completes two operations within the current clock cycle: first, it strips and parses the packet header of the received data packet, extracting the data source identifier field and the data length field; second, it uses the parsed data source identifier as a hardware addressing index and performs a hardware-level matching comparison with the vector data block index stored locally. If a match is successful, it initiates a read request for the corresponding bitmap mask from the local RAM. This design merges data packet parsing and mask retrieval into a single-stage pipelined operation, utilizing hardware parallelism to achieve zero-latency overlap between reception and table lookup. By synchronously completing data packet parsing and mask index matching within a single clock cycle, it avoids the additional latency overhead of traditional two-stage serial processing; directly reusing the data source identifier as a hardware address index eliminates the intermediate steps of address translation and table lookup calculation, minimizing the mask retrieval path; and the design of initiating a read request from local RAM utilizes the low-latency characteristics of on-chip storage, ensuring that the mask data can be returned in time in the next clock cycle for use in the data splitting step.
[0050] In the implementation of this application, in the data splitting step, the bitmap mask is read from the local library according to the extracted index, the corresponding data payload is temporarily stored in the register, and the received vector data stream is matched bit by bit with the bitmap mask according to the preset filtering granularity, so that the data unit with the validity mark of 1 in the bitmap mask is identified as valid data and retained, and the data unit with the validity mark of 0 is identified as invalid redundant data and directly discarded.
[0051] Specifically, during the packet information parsing and matching step, while a mask read request is initiated, the corresponding data payload is temporarily stored in the pipeline register awaiting alignment. Upon entering the data splitting step, the hardware reads the target bitmap mask from the local library and performs a bit-by-bit logical matching operation between each data unit in the vector data stream and the corresponding bit of the mask, according to a preset filtering granularity. The matching rule is: when the mask bit is 1, the data unit at the corresponding position is determined to be valid data and retained to enter the next stage of the pipeline; when the mask bit is 0, the data unit at the corresponding position is determined to be invalid redundant data and directly discarded, not entering the subsequent cache link. This hardware-level physical filtering is completed before the data enters the cache, blocking the propagation of redundant data at the source. The bit-by-bit logical matching mechanism allows the filtering judgment to be completed in parallel at the hardware logic level through a simple AND gate array, processing multiple bits of data in a single clock cycle, matching the filtering throughput with the data bus bandwidth. By directly discarding invalid data at the physical level, redundant data is avoided from unnecessarily occupying cache storage space and memory access bandwidth. This ensures that subsequent cache write and read operations are performed only on valid data, thereby improving the utilization efficiency of cache resources and the proportion of effective bus bandwidth.
[0052] In the embodiments of this application, in the effective data caching step, the effective data stream after the previous cleaning and splitting is received and a write operation is triggered. The address mapping logic no longer follows the original discrete address distribution of the sparse matrix, but generates continuous auto-incrementing addresses. The address compression mapping strategy is used to compactly push the effective elements into the cache line of the target cache.
[0053] After the effective data stream, cleaned and filtered in the first two stages, enters this stage, it is no longer stored according to the discrete physical addresses corresponding to the original non-zero element column indices of the sparse matrix. Instead, a continuously increasing hardware address sequence is generated by address mapping logic. Using an address compression mapping strategy, the filtered effective elements are compactly pushed into consecutive cache lines of the target cache RAM in the order of reception, eliminating the storage space fragmentation problem caused by a large number of zero elements occupying space in the original sparse distribution. This compact storage method ensures that each storage location in the cache line carries effective data, theoretically maximizing cache space utilization.
[0054] In the embodiments of this application, during the effective data caching step, effective data of different types is partitioned and stored in segments, with each segment stored in a different cache block to ensure the alignment of the data's physical address. This provides hardware-level address alignment support for subsequent parallel readings by multiple processing cores. Based on the compact writing of effective data into the cache using address compression mapping, a partitioned and segmented storage strategy is implemented for different types of effective data, storing effective data belonging to different categories or corresponding to different processing cores in different physical cache blocks (RAM blocks). This partitioning method ensures that the starting address of each data partition is aligned with the address boundary required by the hardware, avoiding the additional clock cycle overhead caused by cross-boundary access. Simultaneously, it provides physical address isolation for subsequent parallel readings of the data required by multiple processing cores, preventing address conflicts and cache consistency maintenance overhead during multi-core concurrent access.
[0055] In the embodiments of this application, during the generation of bitmap masks, if all elements in a certain vector data block are not referenced by the non-zero elements of the sparse matrix, the filtering information corresponding to the data block is directly discarded and its corresponding vector data block index is not retained, so as to reduce the storage overhead of the bitmap mask table.
[0056] Specifically, during the process of scanning the sparse matrix and generating the bitmap mask on the host side, if it is detected that all elements in a certain vector data block are not referenced by any non-zero elements, meaning that the data block is completely invalid for subsequent sparse matrix-vector multiplication operations, the system directly skips the mask generation for that data block, does not allocate any filtering information entries for it in the bitmap mask table, and does not retain its corresponding vector data block index. This pruning strategy eliminates the description overhead of completely invalid data blocks from the source of the mask table, compresses the overall storage volume of the mask table, and also reduces the traversal range of the runtime hardware matching index. By directly discarding the filtering information of completely invalid data blocks, the bitmap mask table's footprint in the local storage of the hardware acceleration side is reduced, saving valuable on-chip storage resources; at the same time, the range of indexes that the runtime hardware needs to search and match is narrowed, reducing the complexity of the matching logic, which can improve retrieval efficiency and reduce power consumption in scenarios with large mask table sizes.
[0057] In the embodiments of this application, the hardware acceleration end is an FPGA accelerator, the on-chip data bus width is set to 1024 bits, a single bus transmission can carry 32 32-bit single-precision floating-point numbers, and the bit width of the bitmap mask sequence is 32 bits, corresponding to the number of data units transmitted in a single bus transmission.
[0058] Specifically, the hardware acceleration uses an FPGA as the underlying implementation platform, with the on-chip data bus having a physical width of 1024 bits. This width allows for the parallel transmission of 32 32-bit single-precision floating-point data units in a single bus transaction. To precisely match this hardware transmission granularity, the bitmap mask sequence is correspondingly set to a width of 32 bits, ensuring that each bit of the mask corresponds exactly to the validity marker of a 32-bit floating-point data unit. This bit-width alignment design ensures a one-to-one correspondence between the data payload and the mask, enabling the bit-by-bit parallel matching operation in the data splitting step to process all data units of a single bus transmission within one clock cycle, without the need for batch processing or bit-width conversion. Precisely aligning the bitmap mask width with the number of bus data units allows the data packets and corresponding masks of a single bus transmission to complete all matching operations within the same clock cycle, maximizing the utilization of hardware parallel processing capabilities. The programmable nature of the FPGA platform facilitates flexible adjustment of pipeline depth and memory partitioning strategies according to specific sparse matrix operation scenarios, improving the system's adaptability to different sparsities and data sizes.
[0059] This application takes the sparse matrix operation of three-dimensional myocardial mesh in large-scale cardiac electrophysiological simulation as an example. The underlying hardware platform uses an FPGA accelerator, and the on-chip data bus width is set to 1024 bits (that is, a single bus transmission can carry 32 32-bit single-precision floating-point numbers).
[0060] In the preprocessing stage before the system officially runs, the host first scans the sparse matrix generated by the cardiac electrophysiological simulation and generates a bitmap mask for each FPGA processing core based on the column index of the non-zero elements (i.e., the access address of the source vector).
[0061] Specifically, the host divides the source vector into several vector data blocks based on the data volume of a single bus data packet. For each data block, if an element is referenced by a non-zero element of the sparse matrix during calculation, the corresponding position in the bitmap mask is marked as '1'; otherwise, it is marked as '0'. In particular, if all elements in a vector data block are not referenced, the filtering information corresponding to that data block is discarded, and its corresponding vector data block index is not retained. Subsequently, the host pre-distributes a bitmap mask table containing vector data block indices and data validity markers to the local on-chip storage units of each processing core in the FPGA, preparing data for subsequent rapid hardware retrieval.
[0062] After entering the runtime phase where the FPGA takes over independently, the system runs a three-stage hardware pipeline between the data bus and the source vector cache, such as... Figure 3 As shown.
[0063] When the FPGA processing core receives a source vector data packet from the on-chip bus, the first-level "data packet information parsing and matching" pipeline strips and parses the packet header within the current clock cycle, extracts the data source ID, uses this ID as a hardware addressing index, matches the corresponding vector data block index, and initiates a read request to the local RAM storing the bitmap mask.
[0064] Immediately following, in the next clock cycle, the data stream enters the second-stage "data splitting" pipeline. At this time, RAM returns the corresponding 32-bit wide bitmap mask sequence, while 1024 bits of data are temporarily stored in the pipeline register. The hardware filter performs a parallel logical AND matching operation between this 32-bit mask and the 32 floating-point numbers in the data payload. The floating-point numbers corresponding to a mask of '1' are extracted and retained, while the data corresponding to a mask of '0' are directly identified as invalid and redundant data and discarded.
[0065] Finally, the filtered valid data stream enters the third-stage "valid data cache" pipeline. During this clock cycle, the address mapping logic no longer follows the original discrete address distribution of the sparse matrix, but instead generates continuous auto-incrementing addresses. An address compression mapping strategy is used to compactly push these valid elements into the cache lines of the target RAM. At the same time, different types of valid data are segmented and stored in different RAM blocks to ensure the alignment of the data's physical addresses, providing hardware-level support for the parallel reading of multiple processing cores.
[0066] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.
Claims
1. A bitmap mask filtering and caching method for sparse matrix operations, characterized in that, Includes the following steps: Preprocessing stage: The host generates a bitmap mask based on the spatial distribution characteristics of the non-zero elements of the sparse matrix and pre-downloads it to the local storage of the hardware acceleration end; During runtime: The hardware acceleration module performs real-time validity identification and rejection of the input vector data stream through a three-stage hardware pipeline, which includes: Data packet parsing and matching: Parse the data packet header information and match the corresponding bitmap mask; Data splitting: Perform bit-by-bit matching based on the bitmap mask to filter valid data; Effective data caching: An address compression mapping strategy is used to compactly write effective data into the on-chip target cache.
2. The bitmap mask filtering and caching method for sparse matrix operations according to claim 1, characterized in that, In the preprocessing stage, the host scans the column indices of the non-zero elements of the sparse matrix, divides the source vector into several vector data blocks according to the data volume of a single bus data packet, marks whether the elements in each data block are referenced by non-zero elements, generates a bitmap mask table containing vector data block indices and data validity marks, and pre-distributes the bitmap mask table to the local on-chip storage unit of each computing core on the hardware acceleration side.
3. The bitmap mask filtering and caching method for sparse matrix operations according to claim 2, characterized in that, The bitmap mask is stored in a segmented form, with each segment corresponding to the filtering information of a vector data block. The vector data block index contains the segmentation information of the bitmap mask, and the data validity mark contains the corresponding bitmap mask. 1 and 0 are used to distinguish between vector data that will participate in the operation when multiplying a sparse matrix into a vector and vector data that will not participate in the operation.
4. The bitmap mask filtering and caching method for sparse matrix operations according to claim 2, characterized in that, During the bitmap mask generation process, if all elements in a certain vector data block are not referenced by the non-zero elements of the sparse matrix, the filtering information corresponding to that data block is directly discarded and its corresponding vector data block index is not retained, in order to reduce the storage overhead of the bitmap mask table.
5. The bitmap mask filtering and caching method for sparse matrix operations according to claim 1, characterized in that, The runtime phase is independently managed by the hardware acceleration end. The three-level hardware pipeline splits the data receiving, filtering and diversion, effective extraction, and buffer writing operations into a pipeline level according to clock cycles, so that the entire process delay from data packet input to effective writing is controlled within three clock cycles.
6. The bitmap mask filtering and caching method for sparse matrix operations according to claim 5, characterized in that, In the data packet information parsing and matching step, when the processing core receives the source vector data packet, it synchronously completes the reception of the data packet and the parsing of the packet header, extracts the data source identifier and data length, uses the parsed data source identifier as the hardware address index and matches it with the vector data block index, and retrieves the corresponding bitmap mask from the local information table.
7. The bitmap mask filtering and caching method for sparse matrix operations according to claim 6, characterized in that, In the data splitting step, the bitmap mask is read from the local library according to the extracted index, the corresponding data payload is temporarily stored in the register, and the received vector data stream is matched bit by bit with the bitmap mask according to the preset filtering granularity. This makes the data units with a validity mark of 1 in the bitmap mask recognized as valid data and retained, while the data units with a validity mark of 0 are recognized as invalid redundant data and discarded directly.
8. The bitmap mask filtering and caching method for sparse matrix operations according to claim 7, characterized in that, In the effective data caching step, the effective data stream after the previous cleaning and splitting is received and a write operation is triggered. The address mapping logic no longer follows the original discrete address distribution of the sparse matrix, but generates continuous auto-incrementing addresses. The address compression mapping strategy is used to compactly push the effective elements into the cache line of the target cache.
9. The bitmap mask filtering and caching method for sparse matrix operations according to claim 8, characterized in that, In the effective data caching step, effective data of different types is segmented and stored in different cache blocks to ensure the alignment of the data physical address and provide hardware-level address alignment support for the parallel reading of multiple computing cores in the future.
10. The bitmap mask filtering and caching method for sparse matrix operations according to claim 1, characterized in that, The hardware acceleration end is an FPGA accelerator, with an on-chip data bus width of 1024 bits. A single bus transmission can carry 32 32-bit single-precision floating-point numbers. The bitmap mask sequence has a bit width of 32 bits, corresponding to the number of data units transmitted in a single bus transmission.