Data caching apparatus and method, memory

Abstract

The present disclosure provides a data caching method, which comprises: writing feature map data and weight data read from a dynamic random memory (DRAM) into a second-level cache; and when it is determined that there is free storage space in a first-level cache, writing the feature map data and the weight data in the second-level cache into the first-level cache for reading by an inference module. The present disclosure also provides a data caching device and a memory.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to data caching devices and methods, and memory. Background Technology

[0002] Currently, artificial intelligence is built upon multi-layered, large-scale neural networks, which are essentially modules containing matrix multiplication and convolution operations. Their implementation often requires defining a cost function first, then feeding the read feature maps and weight data into the computational network in batches, deriving the cost function value based on the parameters, and thus updating the entire neural network model; this typically implies extremely high data reading requirements. For example, AI processing algorithms based on high-definition video can require reading up to 2 billion bytes of data just for the feature maps of each convolutional layer. This further congests the already limited DRAM (Dynamic Random Access Memory) bandwidth, and the massive data transfer also means an equivalent amount of energy consumption. Therefore, a pressing need for hardware acceleration devices in deep learning is to accelerate the data transfer process. This can be achieved through dedicated data reading and cache management modules to accelerate data transmission from DRAM to the internal AI computing core, enabling the AI ​​computing core to operate efficiently and continuously, thereby increasing computing power. Summary of the Invention

[0003] This disclosure provides a data caching device and method, and a memory.

[0004] In a first aspect, embodiments of this disclosure provide a data caching method, comprising:

[0005] The feature map data and weight data read from the dynamic random access memory (DRAM) are written into the secondary cache.

[0006] When it is determined that there is free space in the first-level cache, the feature map data and weight data in the second-level cache are written to the first-level cache for the inference module to read.

[0007] In some embodiments, the first-level cache includes a feature map data first-level cache and a weight data first-level cache; the step of writing the feature map data and weight data from the second-level cache into the first-level cache when it is determined that there is free storage space in the first-level cache includes:

[0008] When it is determined that there is free storage space in the first-level cache of feature map data, a first request is sent to the second-level cache to write the feature map data in the second-level cache into the first-level cache of feature map data; when it is determined that there is free storage space in the first-level cache of weight data, a second request is sent to the second-level cache to write the weight data in the second-level cache into the first-level cache of weight data.

[0009] In some embodiments, the secondary cache includes multiple memory bank groups; each memory bank group includes multiple memory banks; the step of writing feature map data and weight data read from dynamic random access memory (DRAM) into the secondary cache includes:

[0010] The feature map data and weight data read from the dynamic random access memory (DRAM) are stored in the corresponding memory blocks of the memory block group according to a preset read / write order and spatial address.

[0011] In some embodiments, the feature map data first-level cache includes multiple cells; each cell includes multiple rooms; the step of sending a first request to the second-level cache to write the feature map data in the second-level cache into the first-level cache when it is determined that there is free storage space in the feature map data first-level cache includes:

[0012] When it is determined that there is a free cell in the primary storage of feature map data, a first request is sent to the secondary cache to write the feature map data in the secondary cache into the free cell's room in the primary cache.

[0013] In some embodiments, the weight data first-level cache includes multiple circular caches, and the number of circular caches is the same as the number of storage units in the second-level cache; when it is determined that there is free storage space in the weight data first-level cache, a second request is sent to the second-level cache to write the weight data in the second-level cache into the first-level cache, the step of which includes:

[0014] When it is determined that the circular cache of the first-level cache of weight data is free, a second request is sent to the second-level cache to write the weight data in the corresponding storage body of the second-level cache into the circular cache.

[0015] In some embodiments, the step of writing the feature map data and weight data read from the dynamic random access memory (DRAM) into the secondary cache includes:

[0016] According to the pre-configured access arbitration logic, the feature map data and weight data read from the dynamic random access memory (DRAM) are written into the secondary cache.

[0017] Secondly, embodiments of this disclosure provide a data caching device, comprising:

[0018] The data read control module is used to read feature map data and weight data from the dynamic random access memory (DRAM) and write them into the secondary cache.

[0019] The second-level cache is used to write the feature map data and the weight data into the first-level cache when there is free storage space in the first-level cache;

[0020] A first-level cache is used to cache the feature map data and the weight data for the inference module to read.

[0021] In some embodiments, the secondary cache includes: a plurality of memory groups, each memory group including a plurality of memory units; wherein,

[0022] Each of the storage groups is used to store the feature map data and weight data in the corresponding storage of the storage group according to a pre-set read / write order and space address.

[0023] In some embodiments, the first-level cache includes:

[0024] The first-level feature map data cache is used to store the feature map data written to the second-level cache;

[0025] The weight data first-level cache is used to store the weight data written to the second-level cache.

[0026] In some embodiments, the feature map data level-one cache includes register units.

[0027] In some embodiments, the feature map data first-level cache includes multiple cells, each cell including multiple rooms; each room is used to store the feature map data.

[0028] In some embodiments, the weighted data first-level cache includes multiple circular caches; each circular cache corresponds to a storage block in a second-level cache.

[0029] Thirdly, embodiments of this disclosure provide a memory including the data cache device described above, and a dynamic random access memory (DRAM).

[0030] The embodiments disclosed herein have the following beneficial effects:

[0031] The data caching device provided in this embodiment includes a data read control module, a first-level cache, and a second-level cache. This data caching device is particularly suitable for core artificial intelligence computations, specifically for storing feature map data and weight data. By reading feature map data and weight data from DRAM through the first-level and second-level caches, the DRAM read bandwidth requirements can be effectively reduced, and the data required for core artificial intelligence computations can be supplied in a timely manner. This saves DRAM access bandwidth and energy consumption, and improves data supply speed and data utilization. Attached Figure Description

[0032] The accompanying drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0033] Figure 1 This is a schematic diagram of feature map storage in DRAM provided by an embodiment of the present disclosure;

[0034] Figure 2 This is a schematic diagram of the structure of a data caching device provided in an embodiment of the present disclosure;

[0035] Figure 3 This is a schematic diagram of a secondary cache structure provided in an embodiment of the present disclosure;

[0036] Figure 4 This is a schematic diagram of another secondary cache structure provided in an embodiment of the present disclosure;

[0037] Figure 5 A schematic diagram of the structure of a first-level cache for feature map data provided in an embodiment of this disclosure;

[0038] Figure 6 A flowchart of a data caching method provided in this disclosure embodiment;

[0039] Figure 7 This is a schematic diagram of the structure of a memory provided in an embodiment of the present disclosure. Detailed Implementation

[0040] To enable those skilled in the art to better understand the technical solutions of the present invention, the data caching device and method and the memory provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0041] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.

[0042] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0043] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0044] The embodiments described herein can be described with reference to plan views and / or cross-sectional views using the ideal schematic diagrams of this disclosure. Therefore, the example illustrations can be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to those shown in the drawings, but include modifications to configurations formed based on manufacturing processes. Therefore, the areas illustrated in the drawings are schematic in nature, and the shapes of the areas shown in the figures illustrate specific shapes of areas of an element, but are not intended to be limiting.

[0045] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0046] It should be noted that the data caching device 1, memory, and data caching method provided in this embodiment are particularly suitable for the model building of neural networks in artificial intelligence; that is, they are particularly suitable for storing feature map data and weight data required by the core of artificial intelligence computation.

[0047] In this embodiment, the inference module refers to the artificial intelligence computing core, and the following description of the embodiments also uses the inference module as the artificial intelligence computing core.

[0048] The feature map data is stored in DRAM (Dynamic Random Access Memory) before being written to the data cache device 1. To clarify how the feature map data is stored in DRAM, the DRAM storage space planning will be explained below.

[0049] Specifically, artificial intelligence extensively utilizes matrix operations. Therefore, to improve reading efficiency, the feature map data required for matrix operations cannot be simply stored linearly according to the spatial position of each element in the feature map. Figure 1As shown, in this embodiment, the feature map is decomposed into small matrix blocks (hereinafter referred to as blk) by combining the specifications of the multiply-accumulate matrix in the artificial intelligence computing core. These blocks are stored contiguously in DRAM in a continuous order of input-channel (hereinafter referred to as ich), as follows: Figure 1 As shown, the feature map is parsed to extract 32 feature map data points from 0 to 31. Each feature data point is further divided into 8 blks from 0 to 7. Each blkich is then written to the corresponding location in the DRAM via its corresponding line (line0, line1, ..., line7). This makes reading the bus more efficient when using burst transfer mode.

[0050] The data caching device 1, the memory, and the data caching method in the embodiments of this disclosure will be described in detail below.

[0051] Firstly, such as Figure 2 As shown in the figure, this disclosure provides a data caching device 1, including: a data read control module 11, a first-level cache 13, and a second-level cache 12. The data read module is used to read feature map data and weight data from DRAM and write them into the second-level cache 12; the second-level cache 12 is used to write feature map data and weight data into the first-level cache 13 when there is free storage space in the first-level cache 13; the first-level cache 13 is used to cache the feature map data and weight data for the core computation of artificial intelligence, so that the inference module can read them.

[0052] It should be noted that the storage space of L1 cache 13 is much smaller than that of L2 cache 12.

[0053] Since the data caching device 1 provided in this embodiment includes a data read control module 11, a first-level cache 13, and a second-level cache 12, and this data caching device 1 is particularly suitable for core artificial intelligence operations, that is, for storing feature map data and weight data. By reading feature map data and weight data from DRAM through the first-level cache 13 and the second-level cache 12, the DRAM read bandwidth requirements can be effectively reduced and the data required for core artificial intelligence operations can be supplied in a timely manner, saving DRAM access bandwidth and energy consumption, and improving data supply speed and data utilization.

[0054] To better understand the functions of the data reading control module 11, the first-level cache 13, and the second-level cache 12 in this embodiment, the following will provide a detailed description of each of these three components.

[0055] The data read control module 11 includes DRAM read bus control logic and read bus arbitration logic. Specifically, the data read control module 11 can read the feature map data and weight data stored in DRAM according to predetermined rules, and transfer the feature map data and weight data to the secondary cache 12.

[0056] like Figure 2 As shown, the L2 cache 12 has a data prefetching function. This L2 cache 12 is composed of SRAM (Static Random Access Memory), and its main function is to prefetch the required data from DRAM into the L2 cache 12 according to the data demand of the artificial intelligence computing core. In this way, the artificial intelligence computing core can obtain the data in time to start the operation, thereby mitigating the DRAM latency effect.

[0057] Regarding the partitioning of the secondary storage, in this embodiment of the disclosure, the secondary cache 12 consists of several storage banks ("storage banks", hereinafter referred to as banks), each bank consisting of an independent SRAM. Several banks can be further encapsulated to form a storage bank group ("storage bank group", hereinafter referred to as a bank-group). Therefore, the secondary storage is divided into multiple bank-groups (bank-group0, bank-group1, bank-group2, ...) for management. Each bank-group stores the contents of all ich values ​​of the current feature map data blk ("all ich values ​​of blk", hereinafter referred to as blk-ai (blk all ich)).

[0058] Specifically, such as Figure 3 As shown, the L2 cache 12 consists of 32 banks, each of which is composed of an independent SRAM. The banks are divided into 4 banks (bank-group0, bank-group1, bank-group2, and bank-group3) for management.

[0059] This configuration is based on the fact that most commonly used convolutional kernels are matrices larger than 1x1. Therefore, when convolving the boundaries of a block (blk), data from adjacent blocks (blk) is required. For example, when convolving the boundaries of a block (blk) in bank-group1 (i.e., when bank-group1 is the main data), the left and right adjacent data of bank-group1, located in bank-group0 and bank-group2, are needed. Each bank-group can release its space after its main data and adjacent data have been read. Once the bank-group space is released, the data prefetching logic can prefetch subsequent block-ai data from DRAM into that space to ensure continuous data transmission and that the AI ​​computing core can always operate at full capacity.

[0060] Similarly, when the convolution kernel is 1x1, since there is no need to read the data of the neighboring bank-group, it is sufficient to directly read the data of one bank-group. This reading mechanism can still be nested in the control logic described above without adding extra overhead.

[0061] For the shared L2 cache 12, both feature map data and weight data are stored in several banks of L2 cache 12. Storage space allocation is similar to traditional data space management mechanisms; for example, a portion of the address space is allocated to feature map data, and another portion is allocated to weight data. The start and end of the address space can be dynamically configured via software. For example: Figure 3 As shown, both feature map data and weight data are stored in 32 banks of the L2 cache 12. Address space 0-511 is allocated to feature map data, and address space 511-1023 is allocated to weight data. The start and end of the address space can be dynamically configured via software. Since the feature map data and weight data share a single L2 cache 12, each bank's write and read ports have a weighted access arbitration logic to ensure that neither feature map data nor weight data monopolizes the use of the L2 cache 12 for an extended period. The weight allocation of each bank's write and read ports in the L2 cache 12 can be dynamically configured via software to allocate more access time to a specific data stream, or to configure a specific data stream as having absolute high priority.

[0062] The reconfigurability of L2 cache 12 is as follows: The cache depth of L2 cache 12 can be dynamically adjusted via software configuration. This adapts to the characteristics of the feature maps and weights data volume of different artificial intelligence neural networks. Furthermore, L2 cache 12 supports a space partitioning management strategy. This ensures that feature map and weight data are read into the cache as much as possible when there is sufficient space in L2 cache 12, and can be dynamically adjusted according to the ich (read / write) situation. Generally, when there is a high ich, the blk-ai data volume is large, so the partitioning of L2 cache 12 is reduced; conversely, when there is a low ich, the partitioning of L2 cache 12 is increased. Each partitioned blk-ai data space has dedicated marking management logic. After completing the configured number of reads, it is marked as free, and new blk-ai data can overwrite this part.

[0063] Specifically, the secondary cache 12 in this embodiment can optionally support a power-of-2 partitioning strategy for the cache space. For example, if the feature map data occupies 512 depths (0-511) of the secondary cache 12 space, and there are 4 bank-groups, it can store 4 blk-ai data for feature maps. If the blk-ai data is very small, one bank-group can store two blk-ai data. Therefore, the 512 space can be configured to be divided in half, resulting in 8 bank-groups, allowing the storage of 8 blk-ai data. In this embodiment, the secondary cache 12 supports up to 128 partitions, therefore, theoretically it can store a maximum of 128 x 4 = 512 blk-ai data for feature maps. Figure 4 As shown, taking the storage space of L2 cache 12 as an example of dividing it into four equal parts, L2 cache 12 can store 4x4=16 blk-ai data of feature maps. Similarly, the space for weight data also supports the above-mentioned equal division management strategy, which will not be described in detail here.

[0064] For the data read / write optimization of the L2 cache 12, the L2 cache 12 in this embodiment can be a single-port or dual-port SRAM; wherein, based on the characteristics of SRAM and DRAM, the overall control rhythm is optimized. First, the read / write order of the 32 banks of the L2 cache 12 can be planned to ensure the overall balance of read and write operations, so as not to block DRAM code or leave the artificial intelligence computing core without data available; then, if a read / write conflict occurs, the aforementioned SRAM read / write arbitration logic will be enabled to arbitrate the conflict and ensure normal access to SRAM. At the same time, the read characteristics of DRAM should be considered, as it will not output data continuously, so SRAM writes will also have certain gaps, which leaves access time for read data when a conflict occurs; in addition, with the space reserved in the L1 cache 13, the continuity of data transmission to the artificial intelligence computing core can be basically guaranteed.

[0065] Specifically, taking feature map data as an example, assuming that L2 cache 12 does not perform power-of-two partitioning, there are 4 bank-groups available. In this case, read-write mutual exclusion control logic can be used to ensure that a bank-group being read is not written to, and a bank-group being written to is not read. Therefore, from a macroscopic perspective, L2 cache 12 can achieve simultaneous read and write functionality. If the power-of-two partitioning function of L2 cache 12 is enabled, there is more space to manage. However, since bank-groups may overlap, in the worst case, there is a 1 / 32 probability of a read-write conflict occurring among the 32 banks. In this case, the aforementioned SRAM read-write arbitration logic will arbitrate the conflict to ensure normal SRAM access. Considering the read characteristics of DRAM, data is not continuously output; therefore, SRAM writes will also have certain intervals, allowing access time for read data in case of conflicts.

[0066] In addition, with the space we reserved in the L1 cache 13, we can basically guarantee the continuity of data transmission to the core of artificial intelligence computing.

[0067] In some embodiments of this disclosure, the first-level cache 13 includes a feature map data first-level cache 131 and a weight data first-level cache 132. The feature map data first-level cache 131 is used to store the feature map data written to the second-level cache 12; the weight data first-level cache 132 is used to store the weight data written to the second-level cache 12.

[0068] The storage space of the feature map data level 1 cache 131 is much smaller than that of the level 2 cache 12. The storage of the feature map data level 1 cache 131 can be composed of hardware shift register units (hereinafter referred to as reg). Specifically, the feature map data level 1 cache 131 in this embodiment can include a number of cells, each cell can contain a number of rooms as needed, and each room contains a number of blks. If the artificial intelligence computing core only needs to start one feature map processing pipeline, then only one cell is enabled; if the artificial intelligence computing core needs to start four feature map processing pipelines in parallel, then four cells will be enabled to provide parallel feature map data output.

[0069] For example: each cell contains 3 rooms, namely the left room (hereinafter referred to as lft-room), the middle room (hereinafter referred to as cnn-room), and the right room (hereinafter referred to as rgt-room), specifically, as follows: Figure 5 As shown, cell 0 includes the left room, the middle room, and the right room. When the convolution kernel size is 1x1, only the data from the cNR-room needs to be read for the AI ​​computation core. When the convolution kernel is larger than 1x1, it is necessary to read the neighbor data for AI computation. Therefore, data from both the LFT-room and RGT-room needs to be read simultaneously. To save Reg resources, the sizes of LFT-room and RGT-room can be smaller than that of cNR-room. This is because LFT-room and RGT-room only store the boundary data of the blk required by the convolution kernel, which only needs to be greater than or equal to half the width of the convolution kernel. Meanwhile, to ensure the buffering capacity of the feature map data's first-level cache (131), each room can store 4 blks corresponding to ichs.

[0070] The storage space of the weight data level 1 cache 132 is also much smaller than that of the level 2 cache 12. The storage body of the level 1 cache 13 in this circle consists of several circular caches; each circular cache corresponds to one bank of the level 2 cache 12. That is, the level 2 storage includes 32 banks, and the number of data items in the cache should also be 32. Because the data caching device 1 of this embodiment needs to be compatible with the specifications of various convolution kernels, the bit width required by the artificial intelligence computing core for the weight data packets will vary depending on the convolution kernel. However, the bit width of our level 2 cache 12 banks is fixed. Therefore, a circular cache is used to adjust the bit width. The smallest unit of this circular cache is a byte, and its working mechanism is consistent with the industry, so it will not be described in detail here.

[0071] Since the data caching device 1 in this embodiment includes a data reading control unit, a secondary storage, a feature map data primary cache 131, and a weight data primary cache 132, and these modules have the functions described above, it achieves progress in efficient reading of feature map and weight data, effectively reduces DRAM read bandwidth requirements and timely supplies the computing data required by the artificial intelligence computing core, saves DRAM access bandwidth and energy consumption, and improves data supply speed and data utilization.

[0072] Secondly, this disclosure also provides a data caching method, which can be implemented using the data caching device 1 described above. For example... Figure 6 As shown, the method in this embodiment specifically includes the following steps:

[0073] S1. Write the feature map data and weight data read from the dynamic random access memory (DRAM) into the secondary cache 12.

[0074] Specifically, in this step, the data reading control module 11 of the data caching device 1 can read the feature map data and weight data in the DRAM, and write the feature data and circle data into the secondary cache 12.

[0075] S2. When it is determined that there is free storage space in the first-level cache 13, the feature map data and weight data in the second-level cache 12 are written into the first-level cache 13 for the artificial intelligence core to read.

[0076] Specifically, in this step, the first-level cache 13 can determine whether to send a request to the second-level cache 12 to obtain the feature map data and weight data in the second-level cache 12 based on the current state of its own storage space.

[0077] In some embodiments, the first-level cache 13 includes a feature map data first-level cache 131 and a weight data first-level cache 132; step S2 above may specifically include the following steps:

[0078] When the feature map data first-level cache 131 determines that its storage space is free, it sends a first request to the second-level cache 12 so that the second-level cache 12 writes the feature map data therein into the feature map data first-level cache 131; when the weight data first-level cache 132 determines that its storage space is free, it sends a second request to the second-level cache 12 so that the second-level cache 12 writes the weight data therein into the weight data first-level cache 132.

[0079] It should be noted that the feature map data written to the feature map data first-level cache 131 will be placed at its port position to facilitate reading by the artificial intelligence core operation; similarly, the weight data written to the weight data first-level cache 132 will also be placed at the port position to facilitate reading by the artificial intelligence core operation.

[0080] In some embodiments, step S1 above may specifically include:

[0081] According to the pre-configured access arbitration logic, the feature map data and weight data read from the dynamic random access memory (DRAM) are written into the secondary cache 12.

[0082] Since the ports of the data read / write control module of the data cache device 1 are pre-configured with feature map data and weight data read / write logic and weight ratio, as mentioned above, they will not be described in detail here; at this time, the data read control module 11 can write the feature map data and weight data read from the dynamic random access memory DRAM into the secondary cache 12 according to the pre-configured access arbitration logic.

[0083] In some embodiments, the secondary cache 12 includes multiple memory bank groups; each memory bank group includes multiple memory banks; step S1 above includes:

[0084] The feature map data and weight data read from the dynamic random access memory (DRAM) are stored in the corresponding memory blocks of the memory block group according to a preset read / write order and spatial address.

[0085] Specifically, since the ports of the data read / write control module of the data cache device 1 are pre-configured with feature map data and weight data read / write logic and weight ratio, as mentioned above and will not be described in detail here; therefore, in this step, the data read / write control module can store the feature map data and weight data in the DRAM into the corresponding memory of the memory group according to the pre-configured read / write order and space address.

[0086] In some embodiments, the feature map data first-level cache 131 includes multiple cells; each cell includes multiple rooms; when it is determined that there is free storage space in the feature map data first-level cache 131, a first request is sent to the second-level cache 12 to write the feature map data in the second-level cache 12 into the first-level cache 13, the step includes: when it is determined that the feature map data first-level cache 131 has free storage space ... Figure 1 When there is a free cell in the primary cache, a first request is sent to the secondary cache 12 to write the feature map data in the secondary cache 12 into the free cell's room in the primary cache 13.

[0087] In some embodiments, the weight data first-level cache 132 includes multiple circular caches, and the number of circular caches is the same as the number of storage bodies in the second-level cache 12. When it is determined that there is free storage space in the weight data first-level cache 132, a second request is sent to the second-level cache 12 to write the weight data in the second-level cache 12 into the first-level cache 13. The step includes: when it is determined that the circular cache of the weight data first-level cache 132 is free, a second request is sent to the second-level cache 12 to write the weight data in the corresponding storage body in the second-level cache 12 into the circular cache.

[0088] Thirdly, such as Figure 7 As shown, this embodiment of the present disclosure provides a memory including the data cache device 1 and the DRAM described above. The data cache device 1 is used to buffer feature map data and weight data in the DRAM for reading by the artificial intelligence core computation.

[0089] Since the feature map data in the DRAM is stored in consecutive addresses in this embodiment, it facilitates the reading by the data read control module 11 in the data cache device 1. Furthermore, since the data cache device 1 includes the data read control module 11, a level 1 cache 13, and a level 2 cache 12, and is particularly suitable for use in core artificial intelligence computations, specifically for storing feature map data and weight data, reading the feature map data and weight data from the DRAM through the level 1 cache 13 and level 2 cache 12 effectively reduces the DRAM read bandwidth requirements and ensures timely supply of data needed for core artificial intelligence computations. This saves DRAM access bandwidth and energy consumption, and improves data supply speed and data utilization.

[0090] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A data caching method, comprising: The feature map data and weight data read from the dynamic random access memory (DRAM) are written into the secondary cache. When it is determined that there is free storage space in the first-level cache, the feature map data and weight data in the second-level cache are written into the first-level cache for the inference module to read. The first-level cache includes a feature map data first-level cache and a weight data first-level cache; The step of writing the feature map data and weight data from the second-level cache into the first-level cache when it is determined that there is free storage space in the first-level cache includes: When it is determined that there is free storage space in the first-level cache of feature map data, a first request is sent to the second-level cache to write the feature map data in the second-level cache into the first-level cache of feature map data; when it is determined that there is free storage space in the first-level cache of weight data, a second request is sent to the second-level cache to write the weight data in the second-level cache into the first-level cache of weight data. The secondary cache comprises multiple memory bank groups; each memory bank group comprises multiple memory banks; the step of writing feature map data and weight data read from dynamic random access memory (DRAM) into the secondary cache includes: The feature map data and weight data read from the dynamic random access memory (DRAM) are stored in the corresponding memory bank of the memory bank group according to the preset read / write order and spatial address. The weighted data first-level cache includes multiple circular caches, and the number of circular caches is the same as the number of storage blocks in the second-level cache; each circular cache corresponds to a storage block in the second-level cache; When it is determined that there is free storage space in the first-level cache for weight data, a second request is sent to the second-level cache to write the weight data from the second-level cache into the first-level cache. The steps include: When it is determined that the circular cache of the first-level cache of weight data is free, a second request is sent to the second-level cache to write the weight data in the corresponding storage body of the second-level cache into the circular cache.

2. The method according to claim 1, wherein, The feature map data first-level cache includes multiple cells; each cell includes multiple rooms; the step of sending a first request to the second-level cache to write the feature map data in the second-level cache into the first-level cache when it is determined that there is free storage space in the feature map data first-level cache includes: When it is determined that there is a free cell in the primary storage of feature map data, a first request is sent to the secondary cache to write the feature map data in the secondary cache into the free cell's room in the primary cache.

3. The method according to claim 1, wherein, The step of writing the feature map data and weight data read from the dynamic random access memory (DRAM) into the secondary cache includes: According to the pre-configured access arbitration logic, the feature map data and weight data read from the dynamic random access memory (DRAM) are written into the secondary cache.

4. A data caching device, comprising: The data read control module is used to read feature map data and weight data from the dynamic random access memory (DRAM) and write them into the secondary cache. The second-level cache is used to write the feature map data and the weight data into the first-level cache when there is free storage space in the first-level cache; A first-level cache is used to cache the feature map data and the weight data for the inference module to read. The first-level cache includes: The first-level feature map data cache is used to store the feature map data written to the second-level cache; The first-level cache for weight data is used to store the weight data written to the second-level cache; The secondary cache comprises: multiple memory bank groups, each memory bank group comprising multiple memory banks; wherein... Each of the storage groups is used to store the feature map data and weight data in the corresponding storage of the storage group according to a preset read / write order and spatial address; The weighted data first-level cache includes multiple circular caches; each circular cache corresponds to a storage volume in a second-level cache.

5. The apparatus according to claim 4, wherein, The feature map data first-level cache includes register units.

6. The apparatus according to claim 4, wherein, The feature map data first-level cache includes multiple cells, and each cell includes multiple rooms; each room is used to store the feature map data.

7. A memory comprising a data cache device as described in any one of claims 4-6, and a dynamic random access memory (DRAM).

Images