A data caching device, integrated circuit and electronic device

Abstract

This application discloses a data caching device, integrated circuit, and electronic device. The device includes: a data block address module configured to parse the operation instructions during the instruction scheduling phase to obtain the operation type and a first data address of the memory device where the operation data is located; and to generate a first data block address and a group start flag based on the first data address and the clock cycles required by the operation instructions; a tag group module configured to generate data tags based on the first data block address and the group start flag; and a data block group module configured to access the memory device based on the first data block address, write the target data block returned by the access to the data cache space, and simultaneously write the associated data tags to the tag cache space. This application parses the operation instructions during the instruction scheduling phase and preloads the operation data from memory. It prefetches complete and continuous data blocks according to the operation requirements, avoiding the fragmented loading defects of traditional caches.

Description

Technical Field

[0001] This application relates to the field of storage, and more particularly to a data caching device, integrated circuit, and electronic device. Background Technology

[0002] In instruction-driven computing architectures (such as AI accelerators and graphics processing units), arithmetic units frequently need to read data from memory (such as DDR SDRAM) when executing instructions. However, such memory suffers from two major bottlenecks: high access latency and high power consumption. To alleviate these problems, traditional solutions add a data cache (such as an SRAM cache) between the arithmetic unit and memory. Its core function is to cache high-frequency data, which not only reduces the power consumption of accessing memory but also effectively reduces access latency and the frequency of memory access conflicts, thereby improving data read speed and the performance of the arithmetic unit to some extent.

[0003] However, traditional solutions are limited by the limited capacity of data caches, insufficient space utilization, and difficulty in handling large-scale parallel data access, and their efficiency in data processing still needs to be improved. Summary of the Invention

[0004] To address the aforementioned problems, this application provides a data caching device, an integrated circuit, and an electronic device.

[0005] In a first aspect, this application provides a data caching device, comprising:

[0006] The data block address module is configured to parse the operation instructions in the instruction scheduling stage to obtain the operation type and the first data address of the memory device where the operation data is located; and generate the first data block address and the group start flag according to the first data address and the clock cycle required by the operation instruction.

[0007] Wherein, each group of the first data block addresses includes the first data block addresses generated in multiple consecutive periods; each group of the first data block addresses is configured with a group start flag;

[0008] The tag group module is configured to generate data tags based on the address of the first data block and the group start flag;

[0009] The data block group module is configured to access the memory device according to the address of the first data block, write the target data block returned by the access to the data cache space, and simultaneously write the associated data tag to the tag cache space.

[0010] Optionally, the data caching device further includes:

[0011] The tag queue module is configured to receive multiple first data block addresses and corresponding group start flags sent by the data block address module within a continuous clock cycle, and temporarily store them in the tag queue according to the receiving time sequence.

[0012] A data block queue module is configured to temporarily store the target data block in response to the memory device returning the target data block;

[0013] Write the target data block in the data queue into the data cache space; simultaneously write the address of the first data block in the tag queue and the group start flag into the tag cache space.

[0014] Optionally, the tag group module includes multiple tag cache spaces, each tag cache space storing one data tag;

[0015] The data block group module includes multiple data cache spaces, each of which stores a target data block;

[0016] The data tags also include validity identifiers, and each validity identifier is associated with a tag cache space and a data cache space;

[0017] The validity identifier is used to verify whether the tag cache space has been written to the first data block address, and whether the data cache space has been written to the corresponding target data block.

[0018] Optionally, the data block address module is configured to generate a first data block address in each clock cycle, and to deduplicate the same first data block address in consecutive clock cycles within each group of first data block addresses;

[0019] The same operation instruction includes multiple sets of the first data block addresses, and each set of the data block addresses contains the same number of addresses.

[0020] Optionally, the data caching device further includes:

[0021] The data address module is configured to obtain at least one second data block address based on multiple first data addresses involved in a single clock cycle;

[0022] The logic judgment module is configured to receive the address of the second data block and determine whether the address of the second data block is contained in the data tag;

[0023] The multiplexing module is configured to select and output the computation data corresponding to the first data address when the address of the second data block is contained in the data tag; otherwise, it waits for the memory device to write the target data block into the data block group.

[0024] Optionally, the data address module is configured to generate multiple second data block addresses and multiple group end flags within the clock cycle required by the operation instruction; each group of second data block addresses is configured with one group end flag; each group of second data block addresses includes multiple second data block addresses;

[0025] The logic judgment module is configured to identify, upon receiving the group end flag, that the target data block corresponding to the address of the second data block in a group has been used.

[0026] Based on the state that a set of target data blocks have been used, new target data blocks are written to their data cache space, and new data tags are written to their corresponding tag cache space.

[0027] Optionally, the logic judgment module is configured to, when it is determined that multiple data tags contain the address of the second data block, obtain the index data tag and the offset of the first data address from the multiple data tags according to the set tag index value;

[0028] The multiplexing module is configured to select and output the computational data corresponding to the first data address based on the index data label and the offset.

[0029] Secondly, this application provides an integrated circuit, comprising:

[0030] At least one data caching device as described in any of the above;

[0031] The instruction scheduling device is configured to send the operation instruction to the data cache device;

[0032] The memory device is configured to return the target data block where the computation data is located in response to a data access request from the data cache device;

[0033] The arithmetic unit is configured to parse the arithmetic instructions, perform arithmetic operations based on the arithmetic data output from the target data block in the data cache unit, and output the arithmetic results.

[0034] Optionally, at least one data caching device includes a first data caching device and a second data caching device; wherein the first data caching device is configured to store target data blocks of a first type of operation data in the same operation type; and the second data caching device is configured to store target data blocks of a second type of operation data in the same operation type.

[0035] Thirdly, this application provides an electronic device comprising: any one of the integrated circuits described above.

[0036] Compared with the prior art, this application has at least one of the following beneficial effects:

[0037] The data caching device provided in this application parses operation instructions (including operation type and data address) during the instruction scheduling phase using an instruction-level prefetching and group caching mechanism, and preloads operation data from memory in advance. Complete, continuous data blocks (such as entire feature map slices) are prefetched according to operation requirements, avoiding the fragmented loading defects of traditional caching. The group caching strategy ensures that there is no splicing delay in the output data, directly matching the pipeline requirements of the operation unit.

[0038] In this application, the tag queue can temporarily store multiple first data block addresses, enabling the data caching device to achieve pipelining of memory access, improve memory bandwidth utilization, support efficient data prefetching mechanisms, and smooth access traffic. This fundamentally solves the bottleneck problem between memory access latency and the data demands of high-parallelism computing units, significantly reducing the waiting time of computing units and improving the overall system performance and energy efficiency.

[0039] Computational units (such as matrix multiply-accumulate arrays) may request the same data block address at different times or due to temporary overlap in instruction parsing. Without a deduplication mechanism, the data cache would send multiple identical read requests to the memory device. Deduplicating data block addresses significantly improves the reuse rate of data blocks in the cache, reduces the number of memory accesses, lowers access latency, and thus speeds up memory access, ultimately improving the data output performance of the data cache. Attached Figure Description

[0040] The accompanying drawings used in the description of the embodiments of this application are briefly introduced below:

[0041] Figure 1 This is a block diagram of an integrated circuit provided in an embodiment of this application;

[0042] Figure 2 This is a block diagram of a data caching device provided in an embodiment of this application;

[0043] Figure 3 This is a block diagram of another data caching device provided in the embodiments of this application;

[0044] Figure 4 This is a block diagram of another integrated circuit provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of data storage in a memory device provided in an embodiment of this application;

[0046] Figure 6 This is a storage schematic diagram of a tag array and a data block array provided in an embodiment of this application;

[0047] Figure 7 This is another storage schematic diagram of the tag array and data block array provided in the embodiments of this application;

[0048] Figure 8 This is another storage schematic diagram of the tag array and data block array provided in the embodiments of this application;

[0049] Figure 9 This is a schematic diagram of the storage of the target data block in the memory device provided in the embodiments of this application. Detailed Implementation

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the specific implementation methods of this application will be described below with reference to the accompanying drawings. The drawings and implementation methods described below are merely some embodiments of this application. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings and implementation methods without creative effort. Any adjustments and improvements made without departing from the concept of this application are within the protection scope of this application.

[0051] To keep the drawings concise, only the parts relevant to this application are shown schematically in each drawing, and they do not represent the actual structure of the product. In addition, to make the drawings concise and easy to understand, some parts with the same structure or function are only shown schematically in some drawings, and there may actually be more or fewer parts with the same structure or function.

[0052] In this application, unless otherwise expressly specified and limited, ordinal numbers, such as "first" and "second," are used only to distinguish and describe related objects, and should not be construed as indicating or implying the relative importance or order between related objects; furthermore, they do not represent the number of related objects. The terms "installation" and "connection" should be interpreted broadly; for example, "installation" can mean direct installation or installation via other components; "connection" can mean direct connection or connection via other components.

[0053] In instruction-driven computing architectures (such as AI accelerators and graphics processing units), arithmetic units frequently need to read data from memory (such as DDR SDRAM) when executing instructions. However, such memory suffers from two major bottlenecks: high access latency and high power consumption. To alleviate these problems, traditional solutions add a data cache (such as an SRAM cache) between the arithmetic unit and memory. Its core function is to cache high-frequency data, which not only reduces the power consumption of accessing memory but also effectively reduces access latency and the frequency of memory access conflicts, thereby improving data read speed and the performance of the arithmetic unit to some extent.

[0054] Currently, data caching devices are used to temporarily store computational data to reduce access to main memory. However, when the computation unit executes computational instructions, the data caching device reads the required computational data from the main memory, which still suffers from high latency. Furthermore, because the cache space is insufficient to hold complete computational blocks, the output data is fragmented into discontinuous segments, requiring multiple loading and reassembly, further impacting the performance of the computation unit. This problem leads to insufficient memory bandwidth utilization, especially in large-scale parallel computing scenarios. Simultaneously, when cache space is insufficient, new data directly overwrites old data areas; the overwrite mechanism actively discards potentially reusable data, particularly during convolution operations. Since the input feature map after the weight sliding window and the input feature map before the sliding window may reside in the same data block, the data block address of the input feature map often overlaps in consecutive clock cycles; this forces a renewed memory access, significantly increasing power consumption and latency.

[0055] The data caching device and integrated circuit provided in this application, through instruction-level prefetching and group caching mechanisms, parses operation instructions (including operation type and data address) during the instruction scheduling stage and preloads operation data from memory in advance; effectively masking the latency of memory access and ensuring that the operation unit (or operation device) can obtain a continuous and stable data supply. Simultaneously, target data blocks are grouped and stored in the data caching device using group start flags. When any target data block within a group is used, overwriting of that group is prohibited, ensuring the reuse of the target data block. Complete, continuous data blocks (such as entire feature map slices) are prefetched according to operation requirements, avoiding the fragmented loading defects of traditional caches. The group caching strategy ensures that there is no splicing delay in output data, directly matching the pipeline requirements of the operation unit. The following description is in conjunction with the accompanying drawings.

[0056] Please refer to Figure 1 This is a structural block diagram of an integrated circuit provided in an embodiment of this application. Figure 1As shown, the integrated circuit includes an instruction scheduling device 100, a data cache device 200, a memory device 300, and a processing device 400. Before the processing device 400 begins executing a computational operation, during the instruction scheduling phase, the necessary computational data is stored in the memory device 300 in the form of data blocks. The computational operations include, but are not limited to, convolution operations, fully connected layer operations, and matrix multiplication operations. The instruction scheduling phase is a preparation and decision-making process, the core of which is parsing instructions and preparing all resources for instruction execution. Its output is a directly executable, resource-ready computational operation. Executing the computational operation is the specific calculation process, the core of which is consuming resources (data and computing power) and producing a definite computational result. The memory device 300 includes multiple physically isolated storage spaces, each configured to store a data block. Each data block contains N consecutively stored data (N≥2), and each storage space is mapped to a unique data block address, which includes the data block address and its offset within the data block. When the memory device 300 responds to a read operation, it accesses the corresponding storage space based on the data block address and outputs the data block stored in that storage space. A cache device is located between the memory device 300 and the arithmetic unit 400. In response to arithmetic instructions sent by the instruction scheduling device 100, it parses the arithmetic instructions, obtains the arithmetic type, the starting address (or base address, or first data block address) of each arithmetic type's data in the memory device 300, and arithmetic-related parameters; based on the starting address, it reads and caches the data block corresponding to the arithmetic data from the memory device 300. When the arithmetic unit 400 responds to an arithmetic instruction sent by the instruction scheduling device 100, the cache device directly provides the arithmetic data to the arithmetic unit 400. When the arithmetic unit 400 needs to reuse the same arithmetic data, the cache device directly provides the arithmetic data to the arithmetic unit 400, which can reduce the frequency of access to the memory device 300.

[0057] In one implementation, the number of data caching devices 200 can be set to multiple. Here, we will illustrate with two data caching devices 200, such as a first data caching device and a second data caching device, each corresponding to a data source of different operation types. For example, in a convolution operation scenario, the first data caching device is configured to process the input feature map, and the second data caching device is configured to process the weights. In another matrix multiplication operation scenario, the first data caching device is configured as the first input feature map, the second data caching device is configured as the second input feature map, and the multiply-accumulate operation array acts as the operation device 400. Within one cycle, it receives multiple first data (input feature map or first input feature map) from the first data caching device and multiple second data (weights or second input feature map) from the second data caching device. The first and second data are multiplied and accumulated in the result of the previous cycle. The calculation result of the multiply-accumulate operation array (operation device 400) is written back to the memory device 300 after post-processing.

[0058] In one implementation, the memory device 300 contains data block addresses and corresponding data blocks, with each address corresponding to one data block. ;in, Represents data block The address is X, where X is the total number of data blocks in memory device 300. Each data block contains multiple data blocks, i.e. Y represents the total number of data blocks contained in each data block. Before an instruction is executed, the mapping between data block addresses and data blocks may be updated; during the execution of an instruction, the mapping between data block addresses and data blocks remains unchanged.

[0059] Please refer to Figure 5 This is a schematic diagram of data storage in a memory device provided in an embodiment of this application. For example... Figure 5 As shown, the data is stored in the memory device 300 in the following manner: , That is, the memory device 300 stores 256 data blocks, each containing 8 data items. Assuming each data item is 1 byte, the size of the memory device 300 is... One implementation of the memory device 300 is a static random access memory (SRAM), configured with a data width of 8 bytes and an address depth of 256. The memory consists of 256 independent banks, each with a capacity of 8 bytes. For example, the address of data 0 includes data block address 0 and offset 0, and the address of data 9 includes data block address 1 and offset 1.

[0060] Figure 2 and Figure 3 A block diagram of a data caching device provided in an embodiment of this application is shown. Figure 4 A block diagram of another integrated circuit provided in an embodiment of this application is shown. It should be noted that: Figure 2 The middle data block address module 201 is equivalent to Figure 4 Module 807 for generating data block addresses Figure 2 The middle tag queue module 211 is equivalent to Figure 4 The Chinese label FIFO 808 Figure 2 The data block queue module 221 is equivalent to Figure 4 Middle data block FIFO 809, Figure 2 The middle label group module 212, the data block group module 222 and Figure 3 The logic judgment module 241 is equivalent to Figure 4 Submodule 802, which includes label groups and data block groups. Figure 4The tag group and data block group submodule 802 includes a tag array 804 (equivalent to data tags in the following description), a data block array 805 (equivalent to target data blocks in the following description), and a hit judgment logic unit 803 (equivalent to logic judgment module 241). Figure 2 Medium memory device 300 is equivalent to Figure 4 Medium memory. Figure 4 The data cache reorganizer 800 is equivalent to Figure 1 Data caching device 200, Figure 4 The instruction scheduling unit is equivalent to Figure 1 Central command scheduling device 100.

[0061] This embodiment provides a data caching device, including:

[0062] The data block address module 201 is configured to parse the operation instructions during the instruction scheduling phase, obtain the operation type and the first data address of the memory device where the operation data is located; and generate the first data block address and group start flag according to the first data address and the clock cycles required by the operation instruction. Each group of first data block addresses includes multiple first data block addresses generated within consecutive cycles; each group of first data block addresses is configured with a group start flag.

[0063] Tag group module 212 is configured to generate data tags based on the address of the first data block and the group start flag.

[0064] The data block group module 222 is configured to access the memory device according to the address of the first data block, write the target data block returned by the access to the data cache space, and simultaneously write the associated data tag to the tag cache space.

[0065] In this embodiment, a single computation instruction needs to process a large amount of data, and the execution cycle can reach thousands to tens of thousands of cycles. When executing a single computation instruction, the corresponding computing device performs various computational operations, such as convolution, fully connected operations, matrix multiplication, etc. A computation instruction includes: the type of operation (e.g., convolution); the first data address of the input feature map, weights, and output feature map in the memory device; and the set of computational parameters (e.g., convolution stride). The computational data required for various computational operations are pre-stored in the memory device in the form of data blocks.

[0066] During the instruction scheduling phase, before executing the operation, the data block address module 201 parses the received operation instruction to obtain the operation type, the first data address of the memory device where the operation data is located, and the operation parameter set. Based on the first data address, a first data block address is generated, and the operation data is read from the memory device and cached based on the first data block address. During the instruction scheduling phase, in response to the operation instruction parsing result, the target data block is immediately read from the memory device. Unlike traditional caching mechanisms that only trigger memory access when a cache miss occurs and require waiting for data to be returned before a hit, this solution significantly improves the hit rate of the data cache reassembler through a pre-fetch mechanism, thereby accelerating the supply of operation data.

[0067] The data block address module 201 generates a first data block address and its corresponding group start flag in each clock cycle; each data block address is accompanied by a group start flag. For a group of data block addresses, only one group start flag is set, where the group start flag indicates that its associated first data block address is the starting address of the data block address group. The data block address module 201 can group the first data block addresses generated in consecutive cycles, supporting both dynamic and static grouping methods. Static grouping can be preset; for example, a set number of first data block addresses can be defined as a group, and this number can be determined based on data multiplexing during the operation. Dynamic grouping is dynamically adjusted according to the operation type and the data multiplexing rules followed by the operation to ensure that the output data remains continuous.

[0068] Upon receiving the first data block address and its corresponding group start flag from the data block address module 201, the tag group module 212 generates a data tag. After the memory device returns the target data block, the data block group module 222 writes the returned target data block into the data cache space and simultaneously writes the associated data tag into the tag cache space. During computation operations, the computation data in the target data block can be read through the data tag index. When any target data block within a group is used, overwriting of that group's target data blocks is prohibited, ensuring that the target data blocks can be used. In this way, efficient writing of new data blocks to the data block array can be ensured, while avoiding the risk of overwriting unused data blocks.

[0069] In one implementation, the data block address module 201 generates a first data block address in each clock cycle and removes duplicates of the same first data block addresses in consecutive clock cycles within each group. For example, during convolution operations, since the input feature map corresponding to the weight sliding window is likely to be located in the same data block as the input feature map corresponding to the weight sliding window before the sliding window, the data block addresses of the input feature map in consecutive clock cycles often overlap. In this case, after grouping the first data block addresses, the data in the same data block within the same group can be reused, eliminating the need to repeatedly request the same data block from memory in each cycle; this reduces the need to access duplicate target data blocks from memory in consecutive cycles. The same operation instruction includes multiple groups of first data block addresses, and each group of data block addresses has the same number of addresses.

[0070] In one implementation, the data block address module 201 generates a group start flag and a data block address during clock cycle t, i.e. ;in, For a data block address, This is the group start flag. When the group start flag is 1, that is... , indicating the address of the first data block The corresponding target data block is the first target data block (or the first data block) in a group of target data blocks. When the group start flag is 0, that is... , indicating the address of the first data block The corresponding target data block is a non-first target data block (or a non-first data block) in a set of target data blocks.

[0071] In one implementation, a function is defined. Its input is a tuple (or data set) consisting of one or more data addresses, and its output is a tuple consisting of the addresses of all data blocks containing the one or more data addresses. The data block addresses are unique and arranged in ascending order. A tuple formed by arranging several data addresses in chronological order within a consecutive clock cycle is called a set of data addresses. , ;

[0072] Where NR is the number of data addresses in each group of data addresses; GrpAddrR g A data address or data address group is a set of data addresses. Within a single instruction, each data address group contains the same number of data addresses, and G represents the total number of data address groups in the current instruction. Each data address group corresponds to a data block address, calculated using the following formula:

[0073] , GrpAddrP g A set of data block addresses or a group of data block addresses. For example, ; For example, This achieves deduplication of data block addresses; by removing duplicate addresses from eight data block addresses, four unique data block addresses are output. Thus, data block addresses are effectively "deduplicated" on the timeline, significantly improving the reuse rate of data blocks in the cache, reducing memory access frequency, lowering access latency, thereby accelerating memory access speed and ultimately improving the data output performance of the data cache device. When duplicate data block addresses exist in consecutive clock cycles, it indicates that the corresponding data block is reusable. In this case, repeated access to the same data block to the memory device should be avoided; instead, the target data block should be accessed only once (single or in batches), and the cached data block should be reused in subsequent cycles. Based on this mechanism, the number of duplicate data block addresses transmitted in consecutive cycles can be significantly reduced. The data block addresses should be the data block addresses that the data addresses belong to in each clock cycle, arranged from smallest to largest.

[0074] In one implementation, when "grouping" all data addresses (or data block addresses) generated within an arithmetic instruction, the grouping method of the data addresses (or data block addresses) must satisfy condition 1. As a preferred embodiment, the grouping method of the data addresses satisfies both condition 1 and condition 2:

[0075] ;

[0076] ;

[0077] Condition 1 limits the number of groups to less than the storage capacity. This ensures that updates to the target data block are written, guaranteeing a stable hit for data address access in each clock cycle. Condition 2 optimizes the access efficiency of the data cache reorganizer. The data address grouping strategy directly affects the overall performance of the cache reorganizer; these two conditions together constitute a performance guarantee mechanism.

[0078] In one implementation, the tag group module 212 (or tag array) and the data block group module 222 (or data block array) constitute the core storage unit of the caching device. The tag group module 212 includes multiple tag cache spaces, each storing one data tag. The data block group module 222 includes multiple data cache spaces, each storing one target data block. Each data tag also includes a validity identifier, with each validity identifier associated with one tag cache space and one data cache space. The validity identifier is used to verify whether the tag cache space has been written to the address of the first data block, and whether the data cache space has been written to the corresponding target data block.

[0079] In one implementation, the "tag array-data block array" can store L rows of "first data block address-target data block" mapping pairs. The tag array TagArr can be represented as:

[0080] ;

[0081] in, That is, each tag is a data block address. This serves as a validity indicator, signifying the validity of the "first data block address - target data block" field in the given line. Marks the beginning of a set of target data blocks. A tag array can store a maximum of L tags.

[0082] The data block array DBArr can be represented as: ;in, A data block array can store a maximum of L target data blocks, with labels in the same row corresponding to target data blocks in the same row. Based on the DBGrp header of the data block group, the target data blocks are dynamically grouped, meaning that the grouping of the target data blocks within the data block array changes dynamically each clock cycle.

[0083] In one implementation, when preloading all data blocks mapped to the first data block address into the data block array, the first data block address and group start flag are temporarily stored in the tag queue module 211 (or tag FIFO). After the target data block corresponding to the first data block address is written into the data block queue module 221 (data FIFO), the "first data block address - target data block" mapping with the group start flag is synchronously stored in the tag array and the data block array. The tag FIFO and data FIFO allow the data caching device to receive and temporarily store multiple first data block addresses sent in multiple consecutive clock cycles according to the receiving timing, even before the memory device has returned all target data blocks. When both the tag FIFO and data FIFO are not empty, the read first data block address and its group start flag are written into the tag array, and the corresponding target data block is simultaneously written into the data block array. The tag FIFO has the same depth as the data FIFO, which is also the cache size of the data caching device. Its depth is configured to be no less than the average memory access latency, which can mask the latency from receiving a data block address to returning data (i.e., memory access latency) in most scenarios, ensuring that data blocks are loaded into the data block array in a timely manner, enabling timely data address hits and improving reassembler performance. Because memory devices have high access latency, the time required to prepare data is much longer than the time to process a single request. If the data caching device only sends a data block address to the memory device each time and waits for the target data block to return, its data path will remain idle for a long time.

[0084] In one implementation, please refer to Figure 6 , Figure 7 , Figure 8 Because the storage capacity of the tag array and data block array is limited, while the amount of data that the computation instructions need to process is enormous, requiring continuous loading of memory data, the data caching device needs to implement an overwrite strategy (or replacement strategy). When the data caching device executes the replacement strategy, a first data block address output by the tag FIFO and a target data block output by the data FIFO, i.e., a "first data block address-target data block" pair, are simultaneously updated and written to the already used tag cache and data block cache in the same row, thereby updating the cached data. Writing is performed in ascending order of row number; after writing is complete, it loops back to the first row to continue writing. Overwrite can only be performed when the target data block group corresponding to the address of the first data block to be overwritten has been used up.

[0085] A FIFO-like control scheme is employed: the write pointer determines the write position of the "first data block address - target data block". Before receiving the operation instruction, both the tag array and the data block array are empty, all vld values ​​are set to 0, and the write pointer points to row 0. Subsequently, the "first data block address - target data block" read from the memory device is sequentially written to row 1, row 2, and so on, with the corresponding vld values ​​synchronously set to 1. In each cycle, the tag FIFO sends not only a first data block address but also a group start flag DBGrp for that first data block address. If DBGrp=1, it indicates that the target data block corresponding to that first data block address is the first block of a group of target data blocks, and the last block of that group is the previous target data block of the next target data block corresponding to DBGrp=1. When all 8 rows of the data block array are filled, the write pointer has returned to row 0. The current data block array has completed the writing of 3 groups of target data blocks. The first target data block of the 4th group of target data blocks (or data block group 3) has been written, but the remaining target data blocks have not yet been written. The next target data block in the 4th group of target data blocks (or data block group 3) will be written to row 0, overwriting the first target data block (or data block 0).

[0086] In one implementation, please refer to [the relevant documentation / reference]. Figure 6 , Figure 7 , Figure 8 , Figure 7 It is data block group 3 and its corresponding data tag overlay. Figure 6 Data block group 0 and its corresponding data label. Figure 8 It is data block group 4 and its corresponding data tag overlay. Figure 7The target data blocks in Group 1 and their corresponding data tags are as follows: Group 1 (or Group 0) includes data block 0 (or data block db0), data block 1 (or data block db1), and data block 2 (or data block db2). Group 2 (or Group 1) includes data block 3 (or data block db3) and data block 4 (or data block db4). Group 3 (or Group 2) includes data block 5 (or data block db5) and data block 6 (or data block db6). Group 4 (or Group 3) includes data block 7 (or data block db7), data block 8 (or data block db8), data block 9 (or data block db9), and data block 10 (or data block db10). Group 5 (or Group 4) includes data block 11 (or data block db11) and data block 12 (or data block db12); as shown... Figure 8 As shown, data block group 4 may contain data blocks db11 and db12, as well as other data blocks. Data block group 0 is currently in use. Because data block group 0 is in use, the second target data block (data block 8) in data block group 3 cannot overwrite data block 0 in data block group 0. Once data block group 0 is no longer in use and data block group 1 is in use, data block 8 is allowed to overwrite data block 0, data block 9 to overwrite data block 1, and data block 10 to overwrite data block 2. If data block group 1 is still in use, data block 11 cannot overwrite data block 3, data block 12 cannot overwrite data block 4, and so on. When a data block group is overwritten, its corresponding data tag is overwritten synchronously, which will not be elaborated here.

[0087] Please continue to refer to this. Figure 3 and Figure 4 It should be noted that: Figure 3 The data address module 231 is equivalent to Figure 4 Data address generation module 801, Figure 2 The 251 multi-channel selector module is equivalent to Figure 4 The data selection submodule 810 includes a multiplexer 806. Figure 2 The medium computing device 400 is equivalent to Figure 4 A parallel computing unit. The data caching device provided in this embodiment also includes:

[0088] The data address module 231 is configured to obtain at least one second data block address based on the plurality of first data addresses involved in a single clock cycle.

[0089] The logic judgment module 241 is configured to receive the address of the second data block and determine whether the address of the second data block is contained in the data tag.

[0090] The multiplexing module 251 is configured to select and output the computation data corresponding to the first data address when the address of the second data block is included in the data tag; otherwise, it waits for the memory device to write the target data block into the data block group.

[0091] In this embodiment, the data address module 231 receives and decodes the operation instruction to obtain the operation type, the first data address of each type of data in the memory device, and the parameters required for the operation. The first data address of the type of data to be read indicates the starting address of that type of data; the operation type and related parameters together determine the reading order of that type of data in subsequent clock cycles.

[0092] Within each clock cycle, the data address module 231 obtains multiple second data block addresses based on the first data addresses of multiple operational data in the memory device and sends them to the logic judgment module 241. The logic judgment module 241 compares the received multiple second data block addresses with all the first data block addresses in the tag group module 212 (or tag array). If a second data block address is the same as a first data block address, it is considered that the second data block address is included in the data tag, which can also be defined as a hit; otherwise, it is a miss.

[0093] In the event of a cache miss, the system waits for the memory device to write the target data block to the data block group module 222 (or the data block array) until a cache miss occurs. In the event of a cache miss, the multiplexing module 251 retrieves multiple data blocks corresponding to the addresses of the second data blocks from the data block group module 222 and outputs the computation data to the computing device. The multiplexing module 251 retrieves data across the entire data block group module 222, unlike traditional caches where the retrieval range is a single data block. This expanded retrieval range ensures that when the computation data is discretely distributed (e.g., in the case of convolution with stride > 1), it is not necessary to retrieve all the computation data in multiple cycles; instead, all the computation data can be retrieved in a single cycle.

[0094] In one implementation, the data address module 231 simultaneously generates a data address group end flag, AddrDataGrp, while outputting each data address, for the tag array and data block array to implement a replacement strategy. Each group of second data block addresses is configured with a group end flag; each group of second data block addresses includes multiple second data block addresses. The data address module 231 generates the data address in clock cycle t.

[0095] , ;

[0096] Where T is the number of clock cycles required for the arithmetic unit to complete the current arithmetic instruction. This is the end-of-data address group marker. When it is 1, it indicates the end of the data address group. This is the last data address (or the end data address) in a set of data addresses. A total of [number] data addresses are generated per clock cycle. Each data address corresponds to the one required by the computing device. Each piece of data has an address of 1. ;in, , ;in, The address of the data block where the data is located. This represents the offset of the data within its data block. The amount of data to be processed. It may differ in each clock cycle, and In practice, the data address module 231 generates N data addresses per clock cycle, each data address having a 1-bit valid flag. When the valid flag is valid, the corresponding data address is considered valid, and the number of data addresses is... Thus, the data address module 231 generates in each clock cycle Data address.

[0097] The generation method is as follows: Based on the location of each piece of data in the memory device, that is, which data block it is located in and its offset within the data block, the data address corresponding to that data can be calculated, and then all data addresses can be calculated. For example... Figure 9 As shown, assuming , , , ;but .

[0098] In one implementation, whether a data block group is in an "occupied" state is determined by the data address group end flag, AddrDataGrp. If no AddrDataGrp = 1 has been received, data block group 0 is in use by default. Each time an AddrDataGrp = 1 is received, the occupancy number increments by 1; that is, receiving 1 indicates that data block group 1 is in use, receiving 2 indicates that data block group 2 is in use, and so on.

[0099] In one implementation, when the logic judgment module 241 determines that multiple data tags contain the address of the second data block, it retrieves the index data tag and the offset of the first data address from the multiple data tags according to the set tag index value. The multiplexing module 251 selects and outputs the computational data corresponding to the first data address based on the index data tag and the offset. The set tag index value is a fixed index value in the sorting, for example, when sorted in ascending order, the minimum index value, the maximum index value, or the second index value is taken. This simplifies the data block identification process.

[0100] In one implementation, data is received from data address module 231. Each data address of Each tag in the tag array Compare, if each data address If they are all contained in the label array, then this A data address is hit, that is...

[0101] ;

[0102] in, Indicates all Data address Hit. In the case of a hit, that is... The logic judgment module 241 calculates the value of each data address. The index of the matched tag is used; if multiple tags are matched, the minimum index is taken. Data blocks corresponding to the same tag must be identical. ;in, Additionally, the set of data address offsets. The logic judgment module 241 will... and The output is sent to the multiplexer module 251 for data selection. If this... The data address was not hit, that is... If the clock cycle logic judgment module 241 does not output, then it will wait.

[0103] In one implementation, the multiplexing module 251 receives the output from the logic judgment module 241. indexes and offset For each index and its corresponding offset ,pass Find a data block in the data block array, and then... Find a single piece of data to be processed within that data block; in this way, all the data to be processed can be obtained. ;in,

[0104] The data selection module will process the data. Output to the processing unit. Note: The meaning and value of the same parameter symbol are the same in all embodiments.

[0105] In one implementation, this embodiment also provides an electronic device, including any of the integrated circuits described in the above embodiments.

[0106] The above division of units is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these units can be implemented by a processor calling software; for example, an electronic device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to realize the functions of each unit. The processor can be, for example, a general-purpose processor, such as a central processing unit (CPU), and the memory can be internal or external to the device. Alternatively, these units can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuit, which can be understood as one or more processors. For example, in some embodiments, the hardware circuit is an application-specific integrated circuit (ASIC), which implements the functions of some or all units by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a programmable logic device (PLD), which can include a large number of logic gates. The logical relationships between the logic gates are configured through a configuration file, thereby realizing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling programs, or entirely through hardware circuits, or partially through processor calling programs with the remaining parts implemented through hardware circuits.

[0107] It should be noted that the above embodiments can be freely combined as needed. The above are only some embodiments of this application. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data caching device, characterized in that, include: The data block address module is configured to parse the operation instructions in the instruction scheduling stage to obtain the operation type and the first data address of the memory device where the operation data is located; and generate the first data block address and the group start flag according to the first data address and the clock cycle required by the operation instruction. Wherein, each group of the first data block addresses includes the first data block addresses generated in multiple consecutive periods; each group of the first data block addresses is configured with a group start flag; The tag group module is configured to generate data tags based on the address of the first data block and the group start flag; The data block group module is configured to access the memory device according to the address of the first data block, write the target data block returned by the access to the data cache space, and simultaneously write the associated data tag to the tag cache space.

2. The data caching device according to claim 1, characterized in that, Also includes: The tag queue module is configured to receive multiple first data block addresses and corresponding group start flags sent by the data block address module within a continuous clock cycle, and temporarily store them in the tag queue according to the receiving time sequence. A data block queue module is configured to temporarily store the target data block in response to the memory device returning the target data block; Write the target data block in the data queue into the data cache space; simultaneously write the address of the first data block in the tag queue and the group start flag into the tag cache space.

3. The data caching device according to claim 1, characterized in that: The tag group module includes multiple tag cache spaces, each tag cache space storing one data tag; The data block group module includes multiple data cache spaces, each of which stores a target data block; The data tags also include validity identifiers, and each validity identifier is associated with a tag cache space and a data cache space; The validity identifier is used to verify whether the tag cache space has been written to the first data block address, and whether the data cache space has been written to the corresponding target data block.

4. The data caching device according to claim 1, characterized in that: The data block address module is configured to generate a first data block address in each clock cycle, and to deduplicate the same first data block address in consecutive clock cycles within each group of first data block addresses; The same operation instruction includes multiple sets of the first data block addresses, and each set of the data block addresses contains the same number of addresses.

5. The data caching device according to any one of claims 1 to 4, characterized in that, Also includes: The data address module is configured to obtain at least one second data block address based on multiple first data addresses involved in a single clock cycle; The logic judgment module is configured to receive the address of the second data block and determine whether the address of the second data block is contained in the data tag; The multiplexing module is configured to select and output the computation data corresponding to the first data address when the address of the second data block is contained in the data tag; otherwise, it waits for the memory device to write the target data block into the data block group.

6. The data caching device according to claim 5, characterized in that: The data address module is configured to generate multiple second data block addresses and multiple group end flags within the clock cycle required by the operation instruction; each group of second data block addresses is configured with one group end flag; each group of second data block addresses includes multiple second data block addresses. The logic judgment module is configured to identify, upon receiving the group end flag, that the target data block corresponding to the address of the second data block in a group has been used. Based on the state that a set of target data blocks have been used, new target data blocks are written to their data cache space, and new data tags are written to their corresponding tag cache space.

7. The data caching device according to claim 5, characterized in that: The logic judgment module is configured to, when it is determined that multiple data tags contain the address of the second data block, obtain the index data tag from the multiple data tags according to the set tag index value, and the offset of the first data address; The multiplexing module is configured to select and output the computational data corresponding to the first data address based on the index data label and the offset.

8. An integrated circuit, characterized in that, include: At least one data caching device as described in any one of claims 1 to 7; The instruction scheduling device is configured to send the operation instruction to the data cache device; The memory device is configured to return the target data block where the computation data is located in response to a data access request from the data cache device; The arithmetic unit is configured to parse the arithmetic instructions, perform arithmetic operations based on the arithmetic data output from the target data block in the data cache unit, and output the arithmetic results.

9. The integrated circuit according to claim 8, characterized in that, At least one data caching device includes a first data caching device and a second data caching device; The first data cache device is configured to store target data blocks of a first type of operation data in the same operation type; the second data cache device is configured to store target data blocks of a second type of operation data in the same operation type.

10. An electronic device, characterized in that, Including the integrated circuit as described in claim 8 or 9.

Images