Data caching device, integrated circuit and electronic equipment

Abstract

The invention discloses a data caching device, an integrated circuit and electronic equipment, and the device comprises a data block address module which is configured to analyze an operation instruction in an instruction scheduling stage to obtain an operation type and a first data address of a memory device where operation data is located; generating a first data block address and a group start flag according to the first data address and a clock period required by the operation instruction; the label group module is configured to generate a data label according to the first data block address and the group start flag; and the data block group module is configured to access the memory device according to the first data block address, write the target data block returned by the access into the data cache space, and synchronously write the associated data label into the label cache space. According to the method, the operation instruction is analyzed in the instruction scheduling stage, and the operation data are pre-loaded from the memory in advance. Complete continuous data blocks are prefetched according to operation requirements, and the fragmentation loading defect of a traditional cache is avoided.

Description

Technical Field

[0001] The present application relates to the field of storage, and in particular to a data cache device, an integrated circuit, and an electronic device. Background Art

[0002] In instruction-driven computing architectures (such as AI accelerators and graphics processors), the arithmetic unit (ALU) frequently reads data from memory (e.g., DDR SDRAM) when executing instructions. However, this type of memory has two major bottlenecks: high access latency and high power consumption. To alleviate these issues, traditional solutions add data buffers (e.g., SRAM caches) between the ALU and memory. Its core function is to cache high-frequency data, not only reducing the power consumption associated with memory accesses, but also effectively lowering access latency and the frequency of memory access conflicts, thereby improving data read speed and ALU performance to a certain extent.

[0003] However, traditional solutions are limited by the limited capacity of data cache, insufficient space utilization, and difficulty in coping with large-scale parallel data access, and their data processing efficiency still needs to be improved. Summary of the Invention

[0004] In order to solve the above problems, the present application provides a data cache device, an integrated circuit and an electronic device.

[0005] In a first aspect, the present application provides a data caching device, comprising: a data block address module configured to parse an operation instruction in an instruction scheduling phase, obtain an operation type and a first data address of a memory device where the operation data is located, and generate a first data block address and a group start flag based on the first data address and a clock cycle required by the operation instruction; Each group of the first data block addresses includes the first data block addresses generated in a plurality of consecutive cycles; each group of the first data block addresses is configured with a group start flag; a tag group module, configured to generate a data tag according to the first data block address and the group start flag; The data block group module is configured to access the memory device according to the first data block address, write the target data block returned by the access into the data cache space, and synchronously write the associated data tag into the tag cache space.

[0006] Optionally, the data caching device further includes: a tag queue module configured to receive the plurality of first data block addresses and the corresponding group start flags sent by the data block address module in consecutive clock cycles, and temporarily store them in a tag queue according to a receiving timing; a data block queue module configured to store the target data block in a data queue in response to the target data block returned by the memory device; write the target data block in the data queue into the data cache space; and write the corresponding first data block address and the group start flag in the tag queue into the tag cache space synchronously.

[0007] Optionally, the tag group module includes a plurality of tag cache spaces, each of which stores a data tag. The data block group module includes a plurality of data cache spaces, each of which stores a target data block. The data tag further includes a validity identifier, each of which is associated with a tag cache space and a data cache space. The validity identifier is used to verify whether the tag cache space has been written with the first data block address and whether the data cache space has been written with the corresponding target data block.

[0008] Optionally, the data block address module is configured to generate a first data block address in each clock cycle, and to remove the same first data block address in consecutive clock cycles in each group of the first data block addresses. The same operation instruction includes a plurality of groups of the first data block addresses, each of which contains the same number of addresses.

[0009] Optionally, the data cache device further includes: a data address module configured to obtain at least one second data block address according to a plurality of first data addresses involved in a single clock cycle; a logic judgment module configured to receive the second data block address and judge whether the second data block address is included in the data tag; a multiplexer module configured to output operation data corresponding to the first data address when the second data block address is included in the data tag; otherwise, wait for the memory device to write the target data block into the data block group.

[0010] Optionally, the data address module is configured to generate a plurality of second data block addresses and a plurality of group end flags in a clock cycle required by the operation instruction; each group of the second data block addresses is configured with a group end flag; and each group of the second data block addresses includes a plurality of second data block addresses. The logic judgment module is configured to identify that a target data block corresponding to a group of the second data block addresses has been completed after receiving the group end flag. Based on a set of target data blocks completed using the state, the new target data block is overwritten in its data cache space, and the new data tag is overwritten in its corresponding tag cache space.

[0011] Optionally, the logic judgment module is configured to, when judging that the plurality of data tags contain the second data block address, obtain an index data tag from the plurality of data tags according to a set tag index value, and the offset of the first data address. The multiplexing module is configured to select and output the operation data corresponding to the first data address according to the index data tag and the offset.

[0012] In a second aspect, the present application provides an integrated circuit, comprising: At least one data cache device as described in any one of the preceding embodiments; An instruction scheduling device configured to send the operation instruction to the data cache device; A memory device configured to return a target data block where the operation data is located in response to a data access request of the data cache device; An operation device configured to parse the operation instruction, output the operation data based on accessing the target data block in the data cache device, perform an operation operation and output an operation result.

[0013] Optionally, the at least one data cache device comprises a first data cache device and a second data cache device; wherein the first data cache device is configured to store a target data block of a first type of operation data in the same operation type; and the second data cache device is configured to store a target data block of a second type of operation data in the same operation type.

[0014] In a third aspect, the present application provides an electronic device, comprising the integrated circuit as described in any one of the preceding embodiments.

[0015] Compared with the prior art, the present application has at least one of the following beneficial effects: The data cache device provided by the present application uses instruction-level prefetching and grouping cache mechanism to parse operation instructions (including operation type and data address) in the instruction scheduling stage, and preloads operation data from the memory. The complete continuous data block (such as the whole feature map slice) is pre-fetched according to the operation requirement, and the defects of traditional cache slicing loading are avoided. The grouping cache strategy ensures that the output data has no splicing delay and directly matches the operation unit pipeline requirement.

[0016] The tag queue in the application can temporarily store a plurality of first data block addresses, so that the data cache device can realize pipelining operation of memory access, improve memory bandwidth utilization, support efficient data prefetching mechanism and smooth access flow. The bottleneck problem between memory access delay and high parallel operation unit data demand is fundamentally solved, the waiting time of the operation unit is significantly reduced, and the performance and energy efficiency ratio of the overall system are improved.

[0017] The operation device (such as a matrix multiplication and addition array) requests the same data block address in different periods or due to the temporary overlap of instruction analysis. Without the deduplication mechanism, the data cache device will initiate multiple identical read data requests to the memory device. After the data block address deduplication, the data block reuse rate in the cache is significantly improved, the memory access times are reduced, the access delay is reduced, the memory access speed is accelerated, and finally the data output performance of the data cache device is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] The drawings used in the description of the embodiments of the application are briefly introduced as follows: Figure 1 is a block diagram of an integrated circuit provided by an embodiment of the application; Figure 2 is a block diagram of a data cache device provided by an embodiment of the application; Figure 3 is a block diagram of another data cache device provided by an embodiment of the application; Figure 4 is a block diagram of another integrated circuit provided by an embodiment of the application; Figure 5 is a schematic diagram of data storage in a memory device provided by an embodiment of the application; Figure 6 is a storage schematic diagram of a tag array and a data block array provided by an embodiment of the application; Figure 7 is another storage schematic diagram of a tag array and a data block array provided by an embodiment of the application; Figure 8 is still another storage schematic diagram of a tag array and a data block array provided by an embodiment of the application; Figure 9 is a storage schematic diagram of a target data block in a memory device provided by an embodiment of the application. DETAILED DESCRIPTION

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, specific implementation manners of the present application will be described below with reference to the drawings. The drawings and implementation manners described in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings and implementation manners can be obtained from these drawings and implementation manners without creative effort, and adjustments and improvements made without departing from the concept of the present application are within the protection scope of the present application.

[0020] For the sake of simplicity of the drawings, only parts related to the present application are shown in the drawings, which do not represent the actual structure of the product. In addition, in order to make the drawings simple and easy to understand, in some drawings, only parts with the same structure or function are shown, and there can be more or less parts with the same structure or function.

[0021] In the present application, unless otherwise explicitly specified and limited, ordinal numbers such as "first", "second", etc. are only used to distinguish the description of the associated objects, and cannot be understood as indicating or implying the relative importance or order between the associated objects; in addition, it also does not represent the number of associated objects. The terms "mounting", "connecting" should be understood broadly, for example, "mounting" can be direct mounting or mounting through other components; "connecting" can be direct connection or connection through other components.

[0022] In instruction-driven computing architectures (such as artificial intelligence accelerators, graphics processors, etc.), the operation unit needs to frequently read data from the memory (such as DDR SDRAM) when executing operation instructions. However, such memory has two major bottlenecks: high access delay and high power consumption overhead. To alleviate the above problems, the traditional scheme adds a data buffer (such as an SRAM cache) between the operation unit and the memory, and its core role is to cache high-frequency data, not only reducing the power consumption overhead caused by accessing the memory, but also effectively reducing the access delay and the frequency of memory access conflicts, to a certain extent, improving the speed of data reading and the performance of the operation unit.

[0023] Currently, the access to the main memory is reduced by temporarily storing the operation data through a data cache device, however, when the operation unit executes the operation instruction, the data cache device reads the operation data required for the current operation from the memory device, and the problem of high delay of the memory device still exists. In addition, due to the insufficient cache space to accommodate the complete operation block, the output data is divided into discontinuous fragments, which needs to be loaded and reorganized multiple times, further causing performance loss of the operation unit. Especially in the large-scale parallel computing scenario, this problem leads to insufficient utilization of memory bandwidth. At the same time, when the cache space is insufficient, the new data directly covers the old data area; the covering writing mechanism actively discards the potentially reusable data, especially in the convolution operation process, since the input feature map corresponding to the weight sliding window before and after the sliding window may be located in the same data block, therefore, the data block address where the input feature map is located in the continuous clock cycle often exists repetition; forced to initiate memory access again, significantly increasing power consumption and delay.

[0024] The data cache device and integrated circuit provided by the application, through the instruction level prefetching and grouping cache mechanism, the operation instruction (including operation type, data address) is analyzed in the instruction scheduling stage, and the operation data is preloaded from the memory; effectively mask the delay of memory access, ensure that the operation unit (or operation device) can obtain continuous and stable data supply. At the same time, the target data block is stored in the data cache device by group start flag, when any target data block in the group is used, the group covering writing is prohibited, ensuring that the target data block can be reused. According to the operation requirement, the complete continuous data block (such as the whole feature map slice) is prefetched, which avoids the defect of traditional cache fragmentation loading. The grouping cache strategy ensures that the output data has no splicing delay, and directly matches the operation unit pipeline requirement. The following will be described with reference to the drawings.

[0025] Please refer to Figure 1 , which is a structure block diagram of an integrated circuit provided by an embodiment of the application. As Figure 1As shown, the integrated circuit includes an instruction scheduling device 100, a data caching device 200, a memory device 300, and an operation device 400. Before the operation device 400 starts to perform an operation operation, the operation data required in the instruction scheduling stage is stored in the memory device 300 in the form of a data block. The operation operation includes but is not limited to convolution operation, fully connected layer operation, and matrix multiplication operation. The instruction scheduling stage is a preparation and decision process, the core of which is to analyze instructions and prepare all resources for instruction execution, and the output result is an operation operation that can be directly executed and resource ready. The execution operation is a specific calculation process, the core of which is to consume resources (data and computing power) and produce a certain operation result. The memory device 300 includes a plurality of physically isolated storage spaces, each of which is configured to store a data block. The data block contains N continuous data (N≥2), and each storage space is mapped to a unique data block address. The data address includes the data block address and the offset in the data block. When the memory device 300 responds to the read operation, the corresponding storage space is accessed based on the data block address, and the data block stored in the storage space is output. The cache device is arranged between the memory device 300 and the operation device 400, and responds to the operation instruction sent by the instruction scheduling device 100 to analyze the operation instruction, obtain the operation type, the starting address (or base address or first data block address) of each operation type data in the memory device 300, and the operation related parameters; according to the starting address, the cache device reads and caches the data block corresponding to the operation data from the memory device 300. When the operation device 400 responds to the operation instruction sent by the instruction scheduling device 100, the cache device directly provides the operation data to the operation device 400. When the operation device 400 needs to reuse the same operation data, the cache device directly provides the operation data to the operation device 400, which can reduce the access frequency of the memory device 300.

[0026] In an implementation manner, the number of data caching devices 200 can be set to multiple, and the number of data caching devices 200 is set to two, for example, a first data caching device and a second data caching device, which correspond to data sources of different operation types respectively. For example, in a convolution operation scenario, the first data caching device is configured to process input feature maps, and the second data caching device is configured to process weights. For another example, in a matrix multiplication operation scenario, the first data caching device is configured as a first input feature map, the second data caching device is configured as a second input feature map, and a multiply-accumulate operation array is configured as the operation device 400. In one period, the multiply-accumulate operation array receives a plurality of first data (input feature maps or first input feature maps) from the first data caching device and a plurality of second data (weights or second input feature maps) from the second data caching device, multiplies the first data and the second data, and accumulates the result to the result of the last period. The calculation result of the multiply-accumulate operation array (operation device 400) is written back to the memory device 300 through a post-processing device.

[0027] In an implementation, the data block addresses in the memory device 300 and the corresponding data blocks, each address corresponding to one data block, are ; wherein represents the address of the data block , and X is the total number of data blocks in the memory device 300. Each data block contains a plurality of data Data, i.e. ; wherein Y is the total number of data contained in each data block. The correspondence between the data block addresses and the data blocks can be updated before the execution of an operation instruction; the correspondence between the data block addresses and the data blocks remains unchanged during the execution of an operation instruction.

[0028] Please refer to Figure 5 , which is a schematic diagram of data storage in a memory device provided by the embodiments of the present application. As shown in Figure 5 , the storage manner of data in the memory device 300 is , . That is, the memory device 300 stores 256 data blocks, each data block containing 8 data, and assuming that the size of each data is 1 byte (Byte), the size of the memory device 300 is . An implementation of the memory device 300 is a static random access memory (SRAM), which is configured as follows: data width of 8 bytes (Byte), addressing depth of 256; the memory is composed of 256 independent memory banks (bank), and the capacity of each memory bank is 8 bytes. For example, the address of data 0 includes data block address 0 and offset 0, and the address of data 9 includes data block address 1 and offset 1.

[0029] Figure 2 and Figure 3 show a block diagram of a data cache device provided by the embodiments of the present application, Figure 4 show a block diagram of another integrated circuit provided by the embodiments of the present application. It should be noted that: Figure 2 the data block address module 201 in Figure 4 corresponds to the data block address generation module 807 in Figure 2 the tag queue module 211 in Figure 4 corresponds to the tag FIFO 808 in Figure 2 the data block queue module 221 in Figure 4 corresponds to the data block FIFO 809 in Figure 2 the tag group module 212, the data block group module 222 and Figure 3 the logical judgment module 241 in Figure 4 corresponds to the tag group and data block group submodule 802 in Figure 4The winning tag group and data block group submodule 802 includes a tag array 804 (equivalent to the data tag described below), a data block array 805 (equivalent to the target data block described below), and hit judgment logic unit 803 (equivalent to the logical judgment module 241), Figure 2 The winning memory device 300 is equivalent to Figure 4 The winning memory. Figure 4 The winning data cache reorganizer 800 is equivalent to Figure 1 The winning data cache device 200, Figure 4 The winning instruction scheduling unit is equivalent to Figure 1 The winning instruction scheduling device 100.

[0030] The embodiment provides a data cache device, which comprises: A data block address module 201 is configured to parse an operation instruction in an instruction scheduling stage to obtain an operation type and a first data address of an operation data in a memory device, and generate a first data block address and a group start flag according to the first data address and a clock cycle required by the operation instruction. Each group of first data block addresses comprises a plurality of first data block addresses generated in consecutive cycles, and each group of first data block addresses is configured with a group start flag.

[0031] A tag group module 212 is configured to generate a data tag according to the first data block address and the group start flag.

[0032] A data block group module 222 is configured to access the memory device according to the first data block address, write a target data block returned by the access into a data cache space, and synchronously write an associated data tag into a tag cache space.

[0033] In the embodiment, a single operation instruction needs to process a large amount of data, and the execution cycle can reach several thousand to several ten thousand cycles. When a single operation instruction is executed, the corresponding operation device performs various operation operations such as convolution, full connection, matrix multiplication, etc. One operation instruction includes: an operation type (such as convolution); first data addresses of input feature maps, weights and output feature maps in the memory device; an operation parameter set (for example, convolution stride). The operation data required by various operation operations is stored in the memory device in the form of data blocks in advance.

[0034] In the instruction scheduling stage, before performing the operation operation, the data block address module 201 parses the received operation instruction to obtain the operation type, the first data address of the operation data in the memory device, and the operation parameter set. The first data block address is generated according to the first data address, and the operation data is read from the memory device and cached according to the first data block address. In the instruction scheduling stage, the operation of the target data block is immediately read from the memory device in response to the operation instruction analysis result. Unlike the traditional cache mechanism, which only triggers memory access when a miss occurs and needs to wait for the data to return before the hit, the pre-reading mechanism of the present scheme significantly improves the hit rate of the data cache reorganizer, thereby accelerating the supply of operation data.

[0035] The data block address module 201 generates a first data block address and its corresponding group start flag in each clock cycle; each data block address is accompanied by a group start flag. A group of data block addresses has and only has one group start flag set. The group start flag indicates that the associated first data block address is the first address of the data block address group. The data block address module 201 can group the first data block addresses generated in consecutive different cycles, supporting both dynamic and static grouping methods. Static grouping can be pre-set; for example, a certain number of first data block addresses are set as a group, and the number can be determined according to the data multiplexing condition in the operation. Dynamic grouping is dynamically adjusted according to the data multiplexing rules followed by the operation type and operation operation to ensure that the output data remains continuous.

[0036] The tag group module 212 generates a data tag when receiving the first data block address and its corresponding group start flag sent by the data block address module 201. After the memory device returns the target data block, the data block group module 222 writes the returned target data block to the data cache space and synchronously writes the associated data tag to the tag cache space. When performing the operation operation, the operation data in the target data block can be read through the data tag index. When any target data block in the group is used, the group target data block is prohibited from being overwritten to ensure that the target data block can be used. In this way, new data blocks can be efficiently written to the data block array, and the risk of overwriting unused data blocks can be avoided.

[0037] In an implementation, the data block address module 201 generates a first data block address in each clock cycle, and the same first data block address in consecutive clock cycles in each group is de-duplicated. For example, in the process of convolution operation, since the corresponding input feature map after the weight sliding window is very likely to be located in the same data block as the corresponding input feature map before the sliding window, the data block addresses of the input feature map in consecutive clock cycles are often repeated. In this case, after the first data block addresses are grouped, the data in the same data block in the same group meet the multiplexing condition, and there is no need to repeatedly request the same data block from the memory in each cycle; the repeated access to the target data block from the memory in consecutive cycles can be reduced. The same operation instruction includes multiple groups of first data block addresses, and the number of addresses in each group of data block addresses is the same.

[0038] In an implementation, the data block address module 201 generates a group start flag and a data block address at clock cycle t, that is, ; wherein, is a data block address, is a group start flag. When the group start flag is 1, that is, , it indicates that the first data block address corresponds to the first target data block (or the first data block) in a group of target data blocks. When the group start flag is 0, that is, , it indicates that the first data block address corresponds to a target data block other than the first target data block (or a data block other than the first data block) in a group of target data blocks.

[0039] In an implementation, a function is defined, the input of which is a tuple (or data group) composed of one or more data addresses, and the output of which is a tuple composed of all data block addresses where one or more data addresses are located. Wherein, the data block addresses are not repeated and are arranged in ascending order. A tuple composed of several data addresses in consecutive clock cycles arranged in time sequence is referred to as a group of data addresses, that is, , ; wherein, NR is the number of data addresses in each group of data addresses; GrpAddrR g is a group of data addresses or data address groups. In an operation instruction, the number of data addresses in each group of data addresses is the same, and G is the total number of groups of data addresses in the current operation instruction. Each group of data addresses corresponds to a group of data block addresses, which is calculated according to the following formula: , ; GrpAddrP g is a group of data block addresses or data block address groups. For example, ; For example, The data block addresses are effectively "deduplicated" in the time axis, which significantly improves the data block reuse rate in the cache, reduces the memory access times, and lowers the access delay, thereby speeding up the memory access and ultimately improving the data output performance of the data cache device. When there are repeated data block addresses in consecutive clock cycles, it indicates that the corresponding data blocks have reusability. At this time, repeated access to the same data block in the memory device should be avoided, and instead, a single access to the target data block (single or batch) should be performed, and the cached data block should be reused in the subsequent cycles. Based on this mechanism, the number of repeated data block addresses transmitted in consecutive cycles can be significantly reduced. The data block addresses should be arranged in ascending order of the data addresses in each clock cycle.

[0040] In an implementation, when grouping all data addresses (or data block addresses) generated in one operation instruction, the grouping manner of the data addresses (or data block addresses) must satisfy condition 1. As a preferred solution, the grouping manner of the data addresses satisfies conditions 1 and 2 at the same time: ; ; Condition 1 is to limit the number of groups to be less than the storage capacity, so as to realize the update of the write target data block and ensure that the data address access in each clock cycle can be stably hit. Condition 2 optimizes the access efficiency of the data cache reorganizer. The grouping strategy of the data addresses directly affects the overall performance of the cache reorganizer, and the above two conditions together constitute a performance guarantee mechanism.

[0041] In an implementation, the tag group module 212 (or tag array) and the data block group module 222 (or data block array) constitute the core storage unit of the cache device. The tag group module 212 includes a plurality of tag cache spaces, each of which stores a data tag. The data block group module 222 includes a plurality of data cache spaces, each of which stores a target data block. The data tag further includes a validity identifier, each of which is associated with a tag cache space and a data cache space; the validity identifier is used to verify whether the tag cache space has been written with the first data block address and whether the data cache space has been written with the corresponding target data block.

[0042] In an implementation, the "tag array-data block array" can store L pairs of "first data block address-target data block" mappings. The tag array TagArr can be represented as: ; wherein, ; i.e. each tag is a data block address, validity identifier, indicating the validity of the row "first data block address - target data block", marks the beginning of a group of target data blocks. The tag array can store up to L tags.

[0043] The data block array DBArr can be represented as: ; where, The data block array can store up to L target data blocks. The tags of the same row correspond to the target data blocks of the same row, i.e. According to the data block group start DBGrp, the target data blocks are dynamically grouped, i.e. in each clock cycle, the grouping of the target data blocks in the data block array is dynamically changing.

[0044] In an implementation, when preloading all data blocks mapped by the first data block address into the data block array, the data block address temporarily stores the first data block address and the group start flag into the tag queue module 211 (or tag FIFO), and after the target data block corresponding to the first data block address is written into the data block queue module 221 (data FIFO), the "first data block address - target data block" with the group start flag is synchronously stored into the tag array and the data block array. Among them, the tag FIFO and the data FIFO allow the data cache device to receive and temporarily store multiple first data block addresses issued in multiple continuous clock cycles according to the receiving time sequence under the condition that the memory device has not returned all target data blocks. When the tag FIFO and the data FIFO are not empty, the read first data block address and its group start flag are written into the tag array, and the corresponding target data block is written into the data block array. The depth of the tag FIFO and the data FIFO is equal, and the depth is also the cache number of the data cache device; the depth value is configured to be not less than the average access delay value of the memory, which can mask the delay of the memory device from receiving the data block address to returning the data (i.e. the access delay of the memory) in most scenarios, ensures that the data block is loaded into the data block array in time, and the data address hits on time and improves the performance of the reorganizer. Because the memory device has a high access delay, the time required for the memory device to prepare data is much longer than the time required for processing a single request. If the data cache device sends only one data block address to the memory device each time and waits for the target data block to return, the data path of the data cache device will be idle for a long time.

[0045] In an implementation, please refer to Figure 6 , Figure 7 , Figure 8Because the storage capacity of the tag array and the data block array is limited, and the amount of data to be processed by the operation instruction is huge, the memory data needs to be continuously loaded, so the data cache device needs to implement an overwrite strategy (or replacement strategy). When the data cache device executes the replacement strategy, a first data block address output by the tag FIFO and a target data block output by the data FIFO, that is, a "first data block address-target data block" pair, are updated at the same time to write the used data block cache and the tag cache in the same row, so as to update the cache data. When writing, the row number is in ascending order; after writing is full, it circulates back to the first row to continue writing. Only when the target data block group corresponding to the first data block address to be overwritten has been used up, can the overwrite be executed.

[0046] FIFO control is adopted: the write pointer determines the writing position of the "first data block address-target data block". Before receiving the operation instruction, the tag array and the data block array are empty, all vld are 0, and the write pointer points to the 0th row. Then, the "first data block address-target data block" read from the memory device is written in the 1st row, the 2nd row, and so on, and the corresponding vld is set to 1 at the same time. The tag FIFO sends a first data block address and the group start flag DBGrp of the first data block address synchronously in each cycle. If DBGrp=1, it indicates that the target data block corresponding to the first data block address is the first block of a group of target data blocks, and the last block of the group of target data blocks is the previous target data block corresponding to the next DBGrp=1 target data block. Among them When the 8 rows of the data block array are all full, the write pointer has returned to the 0th row. The current data block array has completed 3 groups of target data blocks, and the first target data block of the 4th group of target data blocks (or data block group 3) has been written, but the remaining target data blocks have not been written. The next target data block in the 4th group of target data blocks (or data block group 3) will be written in the 0th row, covering the first target data block (or data block 0).

[0047] In an implementation mode, please continue to refer to Figure 6 , Figure 7 , Figure 8 , Figure 7 is data block group 3 and the corresponding data tag covering Figure 6 data block group 0 and the corresponding data tag. Figure 8 is data block group 4 and the corresponding data tag covering Figure 7Data block group 1 and the corresponding data labels. The first group of target data blocks (or data block group 0) includes data block 0 (or data block db0), data block 1 (or data block db1), and data block 2 (or data block db2). The second group of target data blocks (or data block group 1) includes data block 3 (or data block db3) and data block 4 (or data block db4). The third group of target data blocks (or data block group 2) includes data block 5 (or data block db5) and data block 6 (or data block db6). The fourth group of target data blocks (or data block group 3) includes data block 7 (or data block db7), data block 8 (or data block db8), data block 9 (or data block db9), and data block 10 (or data block db10). The fifth group of target data blocks (or data block group 4) includes data block 11 (or data block db11) and data block 12 (or data block db12); as Figure 8 As shown, data block group 4 may contain other data blocks in addition to data blocks db11 and db12. Data block group 0 is currently in use. Since data block group 0 is currently in use, the second target data block (data block 8) in data block group 3 cannot overwrite data block 0 in data block group 0. Once data block group 0 has been used up and data block group 1 is currently in use, data block 8 is allowed to overwrite data block 0, data block 9 overwrites data block 1, and data block 10 overwrites data block 2. If data block group 1 is still in use, data block 11 cannot overwrite data block 3, data block 12 cannot overwrite data block 4, and so on. When a data block group is overwritten, its corresponding data tag is also overwritten, which will not be detailed here.

[0048] Please continue to refer to Figure 3 and Figure 4 , it should be noted that: Figure 3 The data address module 231 is equivalent to Figure 4 In the data address generation module 801, Figure 2 The multi-way selection module 251 is equivalent to Figure 4 The data selection submodule 810 includes a multiplexer 806, Figure 2 The computing device 400 is equivalent to Figure 4 A data cache device provided by this embodiment further includes: The data address module 231 is configured to obtain at least one second data block address according to a plurality of the first data addresses involved in a single clock cycle.

[0049] The logic judgment module 241 is configured to receive the second data block address and determine whether the second data block address is included in the data tag.

[0050] The multiplexing module 251 is configured to select and output the operation data corresponding to the first data address when the second data block address is contained in the data tag; otherwise, wait for the memory device to write the target data block into the data block group.

[0051] In the embodiment, the data address module 231 receives and decodes the operation instruction, obtains the operation type, the first data address of each type of data in the memory device, and the parameters required for the operation. The first data address of the type of data to be read indicates the starting address of the type of data; the operation type and the related parameters jointly determine the reading sequence of the type of data in the subsequent clock cycle.

[0052] In each clock cycle, the data address module 231 obtains a plurality of second data block addresses according to the first data addresses of a plurality of operation data in the memory device, and sends the plurality of second data block addresses to the logical judgment module 241. The logical judgment module 241 compares the plurality of second data block addresses received with all the first data block addresses in the tag group module 212 (or the tag array), and if the second data block address is the same as the first data block address, it is considered that the second data block address is contained in the data tag, which can also be defined as a hit; otherwise, it is a miss.

[0053] In the case of a miss, the memory device is waited to write the target data block into the data block group module 222 (or the data block array) until a hit. In the case of a hit, the multiplexing module 251 retrieves the data blocks corresponding to the plurality of second data block addresses from the data block group module 222, and outputs the operation data to the operation device. The retrieval range of the multiplexing module 251 is the entire data block group module 222, which is different from the retrieval range of a traditional cache which is a data block. The expanded retrieval range is to ensure that when the operation data is discretely distributed (for example, in the case of stride>1 convolution), all operation data can be taken out in one cycle without the need to take out all operation data in multiple cycles and multiple times.

[0054] In one implementation, the data address module 231 synchronously generates a data address group end flag AddrDataGrp while outputting each data address, so as to implement a replacement strategy for the tag array and the data block array. Each group of second data block addresses is configured with a group end flag; each group of second data block addresses includes a plurality of second data block addresses. The data address module 231 generates the data address group end flag AddrDataGrp at clock cycle t when outputting the last data address in the group. , ; Wherein, T is the number of clock cycles required for the operation device to complete the current operation instruction, is the data address group end flag. When it is 1, it indicates that the data address is the last data address (or end data address) in the data address group. N data addresses are generated in each clock cycle corresponding to the required operation data of the operation device, each data address being ; wherein, , , ; wherein, is the data block address where the data is located, is the offset of the data in the data block. The number of operation data may be different in each clock cycle, and , in fact, the data address module 231 generates N data addresses in each clock cycle, each data address having a 1-bit valid flag. When the valid flag is valid, the corresponding data address is considered valid, and the number of data addresses is , thus, the data address module 231 generates data addresses in each clock cycle.

[0055] Generation method: according to the location of each data in the memory device, i.e. which data block it is located in and the offset in the data block, the data address corresponding to the data can be calculated, and all data addresses can be calculated. As shown in Figure 9 , assuming , , , ; then .

[0056] In an implementation, whether the data block group is in the "occupied" state is determined by the data address group end flag AddrDataGrp. If no AddrDataGrp = 1 has been received, it is defaulted that the data block group 0 is in use. With the reception of each AddrDataGrp = 1, the occupation number is increased by 1; that is, 1 indicates that the data block group 1 is in use, 2 indicates that the data block group 2 is in use, and so on.

[0057] In an implementation, when the logic judgment module 241 judges that the plurality of data tags contain the second data block address, the index data tag is obtained from the plurality of data tags according to the set tag index value, and the offset of the first data address. The multiplexer module 251 selects the operation data corresponding to the first data address according to the index data tag and the offset. The set tag index value is the index value of the fixed bit in the sorting, for example, the minimum index value or the maximum index value or the second index value is taken when arranged from small to large. In this way, the data block identification step can be simplified.

[0058] In an implementation, the data address module 231 outputs a plurality of data addresses to the logic judging module 241. The logic judging module 241 compares each data address with each tag in the tag array. If each data address is contained in the tag array, the data address is hit. If not, the data address is miss. The number of hit data addresses is represented as The number of miss data addresses is represented as The number of hit data addresses is represented as The number of miss data addresses is represented as ; The number of hit data addresses is represented as The number of miss data addresses is represented as The number of hit data addresses is represented as The number of miss data addresses is represented as The logic judging module 241 calculates the index of the hit tag of each data address, and takes the minimum value of the indexes if multiple tags are hit. The data block corresponding to the same tag is the same, represented as The offset set of the data address is represented as The logic judging module 241 outputs the index and the offset to the multiplexing module 251. If the data address is miss, the logic judging module 241 does not output. In an implementation, the multiplexing module 251 receives the index and the offset from the logic judging module 241. For each index and the corresponding offset, the multiplexing module 251 finds a data block in the data block array, and finds an operation data in the data block. Thus, all operation data are obtained, represented as The number of operation data is represented as

[0059] The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as The number of operation data is represented as

[0060] The data selecting module outputs the operation data to the operation unit. The same parameter symbol in all embodiments has the same meaning and value.

[0061] In an implementation, the embodiment further provides an electronic device comprising the integrated circuit in any of the above embodiments.

[0062] The division of the above units is only a logical function division, and all or part of them can be integrated into a physical entity or physically separated when actually implemented. In addition, the above units can be implemented in the form of processor calling software; for example, the electronic device includes a processor, the processor is connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or to realize the functions of each unit, wherein the processor is, for example, a general processor, such as a central processing unit (CPU), and the memory is a memory in the device or a memory outside the device. Alternatively, the above units can be implemented in the form of hardware circuit, and the functions of part or all of the units can be realized by designing the hardware circuit, which can be understood as one or more processors; for example, in some embodiments, the hardware circuit is an application specific integrated circuit (ASIC), and the functions of part or all of the units are realized by designing the logical relationship of elements in the circuit; for example, in another implementation, the hardware circuit is a programmable logic device (PLD) that can be implemented, which can include a large number of logic gate circuits, and the logical relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of part or all of the units. All units of the above device can be implemented in the form of processor calling program, or all units can be implemented in the form of hardware circuit, or part of the units can be implemented in the form of processor calling program, and the remaining part can be implemented in the form of hardware circuit.

[0063] It should be noted that the above embodiments can be freely combined as needed. The above is only some embodiments of the present application, and it should be pointed out that those skilled in the art can make some improvements and refinements without departing from the principles of the present application, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A data cache device, characterized in that: include: a data block address module configured to parse an operation instruction in an instruction scheduling phase, obtain an operation type and a first data address of a memory device where the operation data is located, and generate a first data block address and a group start flag based on the first data address and a clock cycle required by the operation instruction; Each group of the first data block addresses includes the first data block addresses generated in a plurality of consecutive cycles; each group of the first data block addresses is configured with a group start flag; a tag group module, configured to generate a data tag according to the first data block address and the group start flag; The data block group module is configured to access the memory device according to the first data block address, write the target data block returned by the access into the data cache space, and synchronously write the associated data tag into the tag cache space.

2. The data cache device according to claim 1, wherein: Also includes: a tag queue module configured to receive the plurality of first data block addresses and the corresponding group start flags sent by the data block address module in consecutive clock cycles, and temporarily store them in a tag queue according to a receiving timing; a data block queue module, configured to respond to the target data block returned by the memory device and temporarily store it in a data queue; The target data block in the data queue is written into the data cache space; and the first data block address corresponding to the tag queue and the group start flag are synchronously associated and written into the tag cache space.

3. The data cache device according to claim 1, wherein: The tag group module includes a plurality of tag cache spaces, each of which stores a data tag; The data block group module includes a plurality of data cache spaces, each data cache space storing a target data block; Wherein, the data tag further includes a validity identifier, and each validity identifier is associated with a tag cache space and a data cache space; The validity identifier is used to verify whether the first data block address has been written into the tag cache space, and whether the corresponding target data block has been written into the data cache space.

4. The data cache device according to claim 1, wherein: The data block address module is configured to generate a first data block address in each clock cycle, and to deduplicate the same first data block addresses in consecutive clock cycles within each group of the first data block addresses; The same operation instruction includes multiple groups of the first data block addresses, and the number of addresses included in each group of the data block addresses is the same.

5. The data cache device according to any one of claims 1 to 4, characterized in that: Also includes: a data address module, configured to obtain at least one second data block address according to a plurality of said first data addresses involved in a single clock cycle; a logic judgment module, configured to receive the second data block address and determine whether the second data block address is included in the data tag; The multi-way selection module is configured to select and output the operation data corresponding to the first data address when the second data block address is included in the data tag; otherwise, wait for the memory device to write the target data block into the data block group.

6. The data cache device according to claim 5, wherein: The data address module is configured to generate a plurality of second data block addresses and a plurality of group end flags within a clock cycle required by the operation instruction; each group of the second data block addresses is configured with one group end flag; each group of the second data block addresses includes a plurality of the second data block addresses; The logic judgment module is configured to, after receiving the group end flag, identify that a group of target data blocks corresponding to the second data block addresses have been completed; Based on the state that a group of target data blocks have been used, new target data blocks are overwritten into their data cache space, and new data tags are overwritten into their corresponding tag cache space.

7. The data cache device according to claim 5, wherein: The logic judgment module is configured to obtain an index data tag and an offset of the first data address from the multiple data tags according to a set tag index value when it is determined that the multiple data tags include the second data block address; The multi-way selection module is configured to select and output the operation data corresponding to the first data address according to the index data tag and the offset.

8. An integrated circuit, characterized in that: include: At least one data cache device according to any one of claims 1 to 7; an instruction dispatching device, configured to send the operation instruction to the data cache device; a memory device configured to return a target data block where the operation data is located in response to a data access request from the data cache device; The computing device is configured to parse the computing instruction, perform computing operations based on accessing the computing data output by the target data block in the data cache device, and output computing results.

9. The integrated circuit according to claim 8, wherein: At least one data cache device includes a first data cache device and a second data cache device; The first data cache device is configured to store target data blocks of first type operation data in the same operation type; the second data cache device is configured to store target data blocks of second type operation data in the same operation type.

10. An electronic device, characterized in that: Comprising an integrated circuit as claimed in claim 8 or 9.

Images