Hierarchical storage system for data stream processors

Abstract

The present application provides a hierarchical storage system for a data stream processor, which includes a register stack respectively set in each processing unit of the data stream processor, a first-level cache shared by a group of processing units, and a second-level cache shared by multiple first-level caches, as well as a first prefetch logic module associated with the register stack and a second prefetch logic module associated with the first-level cache, wherein the first prefetch logic module and the second prefetch logic module pre-set prefetch information to perform prefetch operations on corresponding data. The system can reduce the area of ​​on-chip storage while meeting the memory access bandwidth required by the data stream processing unit.

Description

Technical Field

[0001] The present invention relates to the field of computer architecture, and in particular to a storage system for a data stream processor. Background Art

[0002] To reduce the proportion of control logic within a chip and improve the efficiency of its processing units, a dataflow processor was proposed. Unlike traditional control-flow processors, which execute instructions sequentially, dataflow processors execute instructions as soon as their operands are ready. In other words, as long as there are enough processing units (PEs), all instructions that are not data-dependent can be executed in parallel, greatly improving instruction parallelism. Furthermore, in dataflow processors, data is directly transferred between processing units, significantly reducing the frequency of memory accesses. Therefore, unlike the memory access characteristics of traditional processors, processing units in dataflow processors have high memory bandwidth requirements, and the locality of accessed data is not significant. The traditional multi-level memory hierarchy (multi-level cache plus main memory) is not suitable for dataflow memory. To meet the high bandwidth requirements of the processing units, most dataflow processors use relatively large on-chip memories (e.g., several MB). While this solves the bandwidth problem, it comes at the cost of area. The large number of memory units increases chip manufacturing area and power consumption. Summary of the Invention

[0003] The object of the present invention is to provide a hierarchical storage system for a data stream processor, which can reduce the area of ​​on-chip storage while meeting the memory access bandwidth required by the data stream processing unit.

[0004] The above objectives are achieved through the following technical solutions:

[0005] According to a first aspect of an embodiment of the present invention, a hierarchical storage system for a data stream processor is provided, comprising a register stack respectively arranged in each processing unit of the data stream processor, a first-level cache shared by a group of processing units, and a second-level cache shared by multiple first-level caches, as well as a first prefetch logic module associated with the register stack and a second prefetch logic module associated with the first-level cache, wherein the first prefetch logic module and the second prefetch logic module are used to perform a prefetch operation on the next data block according to the processing status of the current data block by the data stream processor, using corresponding address prefetch information preset in the register stack and the first-level cache, respectively.

[0006] In some embodiments, the preset corresponding address prefetch information is generated during the process of compiling a program executed by the data stream processor, and includes a source data address, a destination address, and a data block size.

[0007] In some embodiments, the system may further include a data preprocessing module configured to generate corresponding address prefetch information of each data block to be used by the program during the process of compiling the program executed by the data stream processor.

[0008] In some embodiments, the data preprocessing module can also be configured to: during the program compilation process, determine the data sparsity of the source data read in, and if it is greater than a set threshold, compress the data; wherein the data sparsity of the source data refers to the proportion of bits with a value of 0 in the source data.

[0009] In some embodiments, the data preprocessing module can also be configured to: divide the data into multiple data blocks according to the program block to which the data belongs, and the size of each data block does not exceed the storage space size of the register push; generate address prefetch information of the data block on the register stack, the first-level cache and the second-level cache according to the data block size and the order of the data blocks; and mark the address prefetch information of the data block corresponding to the next program block in each program block.

[0010] In some embodiments, the first-level cache is accessed using a hash address.

[0011] In some embodiments, the first-level cache and the second-level cache may be implemented using scratchpad memory, and the register file may be implemented using static random access memory.

[0012] According to a second aspect of an embodiment of the present invention, a data stream processor is provided, on which the hierarchical storage system according to the first aspect of an embodiment of the present invention is integrated.

[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0015] Figure 1 A schematic diagram of the structure of a hierarchical storage system for data stream processors according to an embodiment of the present invention is shown.

[0016] Figure 2 A schematic diagram of logical access relationships of a hierarchical storage system for data stream processors according to an embodiment of the present invention is shown.

[0017] Figure 3 A schematic diagram of a data preprocessing process in a hierarchical storage system for a data stream processor according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, it will be appreciated by those skilled in the art that the technical solutions of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0020] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0021] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0022] In a dataflow computing architecture, programs are represented using dataflow graphs. During program compilation, the program is first divided into multiple blocks (also called instruction blocks). A dataflow graph is then formed based on the data dependencies between the blocks. Therefore, a dataflow graph is a graph structure that represents the dependencies between application instructions or instruction blocks. Each node in the dataflow graph represents an instruction, and each edge represents the dependency relationship between one instruction and another. The execution result of each instruction can be directly transferred to another instruction via a dependency edge as its operand. Multiple processing units in a dataflow processor are driven by the dataflow graph. Before execution, instructions or instruction blocks are mapped and transmitted to the corresponding processing units based on the dataflow graph. The dataflow graph is then executed based on the precedence relationship. In existing dataflow processors, processing units have high memory bandwidth requirements. To meet these bandwidth requirements, dataflow processors typically include large on-chip memory, which not only adds additional chip area and power consumption but also presents difficulties in chip manufacturing and implementation.

[0023] In this application, the memory access characteristics of the program in the data flow processor are utilized, and the above-mentioned problems are solved in a combination of software and hardware. In an embodiment of the present application, a hierarchical storage system for a data flow processor is provided, which reduces the area of ​​on-chip storage while meeting the memory access bandwidth requirements of the data flow processing unit. In software, the compiler is used to analyze the sparsity of the data and compress the data to minimize the total amount of data required by the application; at the same time, when the compiler divides the data flow graph, it marks the corresponding data pre-fetch in each program block so that it can be used for subsequent hardware mechanism pre-fetching data. In hardware structure, a hierarchical storage structure is set up, which is set between each level of storage to perform data-oriented pre-fetching, reduce the storage capacity of each layer, and ensure data bandwidth. In addition, a hash address hashing mechanism is set up to reduce the probability of data conflict to ensure the utilization of data bandwidth and processing units.

[0024] Figure 1 The schematic diagram of the structure of a hierarchical storage system for a data stream processor according to an embodiment of the present invention is shown. The system includes a register file (also called a register file, a zero-level cache L0) set separately in each PE of the data stream processor, a first-level cache L1 shared by a group of PEs, and a second-level cache L2 shared by multiple first-level caches L1. The register file is an array of multiple registers, which can usually be implemented by a fast static random access memory (SRAM). Figure 1As shown, the register file has multiple ports, allowing multiple functional units in the PE to concurrently access different registers within it, thereby meeting the data supply needs of functional components with high bandwidth requirements. Both the L1 cache and the L2 cache can be implemented using, for example, a scratch pad memory (SPM).

[0025] In the embodiment of the present invention, corresponding pre-fetch logic (which can also be understood as a functional module for data pre-fetching) is set between each level of cache in the hierarchical storage system, which is used to put the data stored in the next level cache into the corresponding position in the previous level cache in advance according to the pre-set pre-fetch control information. The logical access relationship between the caches at each level in the hierarchical storage system is as follows: Figure 2 As shown. The prefetch logic is closely connected with each level of storage. For example, the prefetch logic associated with L0 for prefetching data from L1 is set in each PE respectively; while the prefetch logic associated with L1 and L2 can be a control logic independent of each PE. The granularity of prefetched data can be different between different caches. For example, between L1 and L2, the granularity of prefetched data can be 32 bits * 128 data each time; between L0 and L1, the granularity of prefetched data can be 256 bits * 4 data each time. This article does not impose any restrictions on this.

[0026] Each L0 cache, located on each processing element (PE) of the data flow processor, is designed to quickly respond to data requests within the processing element. It also includes prefetch logic and corresponding address prefetch information. Before the PE's current data block is about to be executed, the L0 prefetch logic can use the address prefetch information to preemptively request data from the L1 cache, enabling seamless access to the next data block.

[0027] Similarly, the L1 cache uses its prefetch logic to send data requests to the secondary cache L2 in advance according to the address prefetch information, so as to prefetch part of the data stored by the L2 into the L1 cache. In an embodiment of the present invention, the L1 cache can be shared by a group of PEs in the data flow processor, such as Figure 1 The L1 cache shown is shared by 4 PEs. Therefore, L1 needs to support simultaneous parallel access by multiple different PEs, but this may cause conflicts in data access between different PEs. Preferably, in order to reduce the contention conflicts caused by multiple PEs accessing L1 at the same time, when the data in the L2 cache is placed in the L1 cache, the address hashing method can be used to hash the adjacent data to avoid data conflicts as much as possible. Alternatively, the hash mechanism can be set in the access logic of the L1 cache so that the addresses of reading and writing L1 must be hashed to form a hash address before reading and writing L1, thereby maximizing the hashing of the data address.

[0028] Continue to refer Figure 2 The L2 cache, the last level of the hierarchical storage system, stores data that can be directly accessed from external storage (such as RAM) to the L2 cache before the data stream processor begins execution. This data is generated by a program compiled by a compiler and accessed directly from the L2 cache by an external controller or dedicated direct transfer component (such as DMA). This allows for hardware-based data prefetching between storage layers, reducing the amount of storage per layer while ensuring data bandwidth.

[0029] In an embodiment of the present invention, the address prefetch information stored at each level of the hierarchical storage system is generated during program compilation. Therefore, in addition to the aforementioned hierarchical storage structure consisting of the three-level caches L0, L1, and L2, the hierarchical storage system also includes a data preprocessing module at the software level. This module can be integrated into the compiler corresponding to the data stream processor as an independent functional module. Figure 3 The data preprocessing process diagram using the data preprocessing module is given. During the program compilation process, the data preprocessing module can compress the data according to the characteristics of the data in the current program and generate address prefetch information for each level of storage.

[0030] More specifically, when the program is compiled, the sparsity of the source data read in is first analyzed according to the set sparsity threshold. For example, by analyzing the proportion of bits with a value of 0 in the source data, the data sparsity is obtained, and by comparing it with a pre-set sparsity threshold (for example, 80% or other, set according to the characteristics of the specific application or actual needs), it is determined whether the source data needs to be compressed. If the data sparsity of the source data is greater than or equal to the sparsity threshold, it is compressed according to the preset compression algorithm to reduce the total amount of data. The compression algorithm here can adopt existing data compression technology, which is not limited in this article.

[0031] Next, for the source data that has undergone the above compression processing or does not need to be compressed, the data is divided according to the program blocks to which it belongs. The data of each program block is assigned to a data block, and the size of the data block does not exceed the data block threshold (generally set to the same size as L0). That is, the size of each database does not exceed the size of the L0 cache. The data block corresponding to each program refers to the set of input data required by the current program block. When the program is executed, the first data address to be accessed will be calculated based on the memory access address calculated by the instructions in the program. Based on this data address, the data block to be accessed can be matched (the case when accessing the data for the first time); or when accessing the data block, each data block has corresponding pre-fetch information, and the next operation is performed according to the pre-fetch information (when accessing the data subsequently).

[0032] Then, for the data blocks after block division, address planning is performed. According to the data usage in the program, the size of the data block, the order of the data blocks, the size of the cache L0, L1 and L2 at each level, etc., the address prefetch information of the data block in each level of cache is generated. The format of the address prefetch information is shown in the following table:

[0033] Data address prefetch information

[0034] Source data address Destination Address Data block size

[0035] At the same time, during the compilation process, the address of the data that the next program block needs to use can be marked in the program block according to the order in which the program block uses the data.

[0036] Finally, the instruction, data block and data block address prefetch information are transmitted as result data to the corresponding storage structure on the chip.

[0037] When the data stream processor is initialized, it goes through a warm-up phase (for example, initially loading some instructions, data blocks, and prefetch information through DMA and cache mechanisms). Basic data and its prefetch information will be stored in L0 and L1. During execution, the prefetch logic of L0 and L1 will load the next data block based on the size and execution status of the current data block before the current data block is completed. At the same time, for L1 shared by multiple PEs, a hash address mechanism can be used to reduce data conflicts when different PEs access L1 simultaneously.

[0038] In another embodiment of the present invention, a data stream processor is provided, on which the above-mentioned Figure 1 and 2 The hierarchical storage system introduced above will pre-place the data stored in the next-level cache into the corresponding position in the previous-level cache according to the pre-set prefetch control information.

[0039] References in this specification to "various embodiments," "some embodiments," "one embodiment," or "an embodiment" refer to a particular feature, structure, or property described in connection with the embodiment being included in at least one embodiment. Thus, the appearances of the phrases "in various embodiments," "in some embodiments," "in one embodiment," or "in an embodiment" in various places throughout this specification do not necessarily refer to the same embodiment. Furthermore, particular features, structures, or properties may be combined in any suitable manner in one or more embodiments. Thus, particular features, structures, or properties shown or described in connection with one embodiment may be combined, in whole or in part, with features, structures, or properties of one or more other embodiments without restriction, as long as the combination is not illogical or inoperable.

[0040] In this specification, the expressions "including," "having," and similar terms are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. "A" or "an" does not exclude a plurality of cases. In addition, the elements in the drawings of this application are for illustrative purposes only and are not drawn to scale.

[0041] Although the present invention has been described through the above embodiments, the present invention is not limited to the embodiments described herein, and includes various changes and modifications that may be made without departing from the scope of the present invention.

Claims

1. A hierarchical storage system for a data stream processor, comprising a register file provided in each processing unit of the data stream processor, a level 1 cache shared by a group of processing units, and a level 2 cache shared by multiple level 1 caches, and a first prefetch logic module associated with the register file and a second prefetch logic module associated with the level 1 cache, wherein the first prefetch logic module and the second prefetch logic module are configured to prefetch the next data block based on the processing status of the current data block by the data stream processor using corresponding address prefetch information preset in the register file and the level 1 cache, respectively. The system further includes a data preprocessing module, which is configured to: During the compilation of a program executed by a data stream processor, corresponding address prefetch information of each data block to be used by the program is generated; as well as During the program compilation process, the data sparsity of the source data read in is determined. If it is greater than the set threshold, the data is compressed; The data sparsity of the source data refers to the proportion of bits with a value of 0 in the source data.

2. The system according to claim 1, wherein the preset corresponding address prefetch information is generated during the process of compiling a program executed by the data stream processor, and includes a source data address, a destination address, and a data block size.

3. The system according to claim 1, wherein the data preprocessing module is further configured to: Divide the data into multiple data blocks according to the program block to which the data belongs, and the size of each data block does not exceed the storage space size of the register push; Generate address prefetch information of the data block on the register file, the first level cache and the second level cache according to the data block size and the order of the data blocks; and In each program block, the address prefetch information of the data block corresponding to the next program block is marked.

4. The system according to claim 1, wherein the first-level cache is accessed by using a hash address. 5 . The system according to claim 1 , wherein the first-level cache and the second-level cache are implemented using scratch-note memory, and the register file is implemented using static random access memory.

6. A data stream processor, integrated with the hierarchical storage system according to any one of claims 1 to 5.

Images