Cache read-write method and device and artificial intelligence chip

Abstract

The application provides a cache read-write method and device and an artificial intelligence chip, and relates to the chip design and manufacturing technical field.The application comprises the following steps: determining a data loading mode of a current read-write instruction in a level one cache; determining a target cache line based on the data loading mode, a meta information table and an idle cache line table; and acquiring data corresponding to the current read-write instruction based on the target cache line; wherein the level one cache comprises a plurality of main cache lines and a plurality of auxiliary cache lines; the meta information table is used for recording data marks, cache line indexes and auxiliary cache line enabling identifiers; the number of the data marks corresponds to the number of the main cache lines; and the idle cache line table is used for recording cache line indexes of cache lines in an idle state.The method and device provided by the application improve the read-write efficiency of the level one cache, improve the execution efficiency of a computing unit and improve the performance of the artificial intelligence chip.

Description

Technical Field

[0001] This invention relates to the field of chip design and manufacturing technology, and in particular to a cache read / write method, apparatus, and artificial intelligence chip. Background Technology

[0002] In the design of artificial intelligence chips, caches can be directly integrated into the chip using semiconductor processes, forming a multi-level cache architecture. The caches within the chip typically include a Level 1 cache (L1 cache) and a Level 2 cache (L2 cache). The Level 1 cache is directly embedded within the computing unit, storing the most frequently accessed instructions and data, resulting in a high hit rate. The read / write efficiency of the Level 1 cache directly determines the execution efficiency of the computing units within the artificial intelligence chip.

[0003] Therefore, improving the read and write efficiency of the first-level cache has become a technical problem that the industry urgently needs to solve. Summary of the Invention

[0004] This invention provides a cache read / write method, apparatus, and artificial intelligence chip to solve the technical problem of how to improve the read / write efficiency of the first-level cache.

[0005] This invention provides a cache read / write method, comprising: Determine how the current read / write instruction loads data from the L1 cache; Based on the data loading method, metadata table, and free cache line table, the target cache line is determined; Based on the target cache line, obtain the data corresponding to the current read / write instruction; The first-level cache includes multiple primary cache lines and multiple secondary cache lines; the metadata table is used to record data tags, cache line indexes, and secondary cache line enable identifiers; the number of data tags corresponds to the number of primary cache lines; the idle cache line table is used to record the cache line indexes of cache lines that are in an idle state.

[0006] In some embodiments, determining the target cache line based on the data loading method, the metadata table, and the free cache line table includes: When the data loading method is enabled by first-level caching, the data cache corresponding to the current read / write instruction is determined to be in the main cache line of the first-level cache; Match the data tag corresponding to the current read / write instruction with the data tag in the metadata table; The target cache line is determined based on the cache line index corresponding to the matched data tag.

[0007] In some embodiments, determining the target cache line based on the data loading method, the metadata table, and the free cache line table includes: When the data loading method is to bypass the first-level cache, it is determined that the data cached for the current read / write instruction is in the second-level cache or memory; Based on the free cache line table, determine the current secondary cache line; The current secondary cache line is determined as the target cache line; Delete the cache line index of the current secondary cache line from the free cache line table.

[0008] In some embodiments, the method further includes: Based on the free cache line table, it is determined that there are no secondary cache lines in an idle state; Based on the aforementioned free cache line table, determine the current main cache line; The current main cache line is determined as the target cache line; Delete the cache line index of the current main cache line from the free cache line table.

[0009] In some embodiments, the method further includes: The current read / write instruction is determined to be a burst transmission instruction; It is determined that the data corresponding to the current read / write instruction is cached in the second-level cache or in memory; The burst transmission length of the burst transmission instruction is determined to be greater than the maximum burst transmission length supported by the first-level cache; the maximum burst transmission length supported by the first-level cache is the data cache size of one main cache line; Based on the free cache line table, a first primary cache line and a first secondary cache line are determined; the sum of the data cache size of the first primary cache line and the first secondary cache line is equal to the burst transmission length of the burst transmission instruction; Based on the first primary cache line and the first secondary cache line, obtain the data corresponding to the current read / write instruction.

[0010] In some embodiments, after determining that the burst transmission length of the burst transmission instruction is greater than the maximum burst transmission length supported by the L1 cache, the method further includes: Based on the free cache line table, it is determined that there are no secondary cache lines in an idle state; The current read / write instruction is split into multiple sub-read / write instructions; the burst transmission length of the sub-read / write instructions is equal to the maximum burst transmission length supported by the first-level cache; Based on the free cache line table, multiple second main cache lines are determined; the number of second main cache lines is equal to the number of sub-read / write instructions. Based on the multiple second main cache lines, the data corresponding to the current read / write instruction is obtained.

[0011] This invention provides a cache read / write device, comprising: The loading method determination module is used to determine the data loading method in the L1 cache for the current read / write instruction; The cache line determination module is used to determine the target cache line based on the data loading method, the metadata table, and the free cache line table. The cache read / write module is used to obtain the data corresponding to the current read / write instruction based on the target cache line; The first-level cache includes multiple primary cache lines and multiple secondary cache lines; the metadata table is used to record data tags, cache line indexes, and secondary cache line enable identifiers; the number of data tags corresponds to the number of primary cache lines; the idle cache line table is used to record the cache line indexes of cache lines that are in an idle state.

[0012] This invention provides an artificial intelligence chip, including a streaming processor cluster; the streaming processor cluster includes multiple computing units; each computing unit is provided with a level-one cache; the computing unit executes the cache read / write method.

[0013] The present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cache read / write method when executing the computer program.

[0014] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the cache read / write method described above.

[0015] The cache read / write method, apparatus, and artificial intelligence chip provided in this invention determine the data loading method of the current read / write instruction in the L1 cache; determine the target cache line based on the data loading method, metadata table, and free cache line table; and obtain the data corresponding to the current read / write instruction based on the target cache line. The metadata table records data tags, cache line indexes, and secondary cache line enable flags; the free cache line table records the cache line indexes of cache lines in an idle state. Since a primary cache line and a secondary cache line are set in the L1 cache, the number of data tags is equal to the number of primary cache lines, enabling flexible mapping and normal read / write operations on the L1 cache. By setting secondary cache lines, cache line resources can be flexibly supplemented under different data loading methods, allowing the data tags and primary cache lines to maintain a flexible mapping, reducing the mutual influence between read / write instructions loading data from outside the L1 cache and read / write instructions loading data from inside the L1 cache, improving the read / write efficiency of the L1 cache, improving the execution efficiency of the computing unit, and improving the performance of the artificial intelligence chip. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the architecture of the artificial intelligence chip provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the cache line provided by the present invention.

[0020] Figure 3 This is one of the flowcharts of the cache read / write method provided by the present invention.

[0021] Figure 4 This is a schematic diagram illustrating the mapping between data identifiers, primary cache lines, and secondary cache lines provided by the present invention.

[0022] Figure 5 This is one of the flowcharts illustrating burst transmission provided by the present invention.

[0023] Figure 6 This is the second schematic diagram of the burst transmission process provided by the present invention.

[0024] Figure 7 This is a schematic diagram of the cache read / write device provided by the present invention.

[0025] Figure 8 This is a schematic diagram of the structure of the artificial intelligence chip provided by the present invention.

[0026] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, units, or modules is not necessarily limited to those explicitly listed, but may include other steps, units, or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0029] ‌ Figure 1 This is a schematic diagram of the architecture of the artificial intelligence chip provided by the present invention, as shown below. Figure 1 As shown, the artificial intelligence chip 100 includes at least one stream processing cluster (SPC) 110, a second-level cache (L2 cache) 120, and high-bandwidth memory 130. The stream processing cluster 110 includes multiple computing units (CUs) 111 and a cluster bus interface (CBI) 112. Each computing unit has a first-level cache (L1 cache) 113.

[0030] The multi-level cache architecture consists of a level 1 cache, a level 2 cache, and high-bandwidth memory. The level 1 cache and level 2 cache are connected via a cluster bus.

[0031] When a computing unit needs to read data, it searches for and retrieves the data in the following order: First, it checks the nearest L1 cache. If the data exists there, it is read directly, which is the fastest access method. If the data is not in the L1 cache, it searches the L2 cache via the cluster bus. If the data is found in the L2 cache, it is read from there. If the required data is not found in the L2 cache either, it is finally read from high-bandwidth memory. This is the slowest access level but has the largest storage capacity.

[0032] Therefore, it can be seen that the read and write efficiency of the L1 cache directly determines the execution efficiency of the computing units in the artificial intelligence chip.

[0033] A cacheline is the smallest unit of data storage and management in the first-level cache, and its core function is to improve system performance by optimizing memory access patterns.

[0034] Figure 2 This is a schematic diagram of the cache line provided by the present invention, as shown below. Figure 2As shown, the L1 cache of a certain chip includes multiple cache lines, each of which can be used to cache a fixed size of data. The size of the cache line is determined according to the chip architecture, for example, 256 bytes.

[0035] Typically, three pieces of information are used to describe the location of data within a cache line: the data tag, the cache line index, and the offset. The high-order bits of the memory address are used as the data tag to uniquely identify the data. The cache line index determines the location of the cache line in the L1 cache. The offset locates the specific position of the data within the cache line. Valid bits represent the valid data.

[0036] The mapping between data tags and cache lines is not fixed (e.g., Tag 0 corresponds to Cacheline 0), but rather flexible (e.g., Tag 0 can be mapped to any Cacheline). When mapping is needed, the cache line index of the Cacheline corresponding to the Tag is recorded in the metadata table.

[0037] The cache uses a set-associative structure, which divides the cache into multiple sets, each set containing a fixed number of ways, and each way containing one cache line. A specific memory address can be mapped to any way within a set. This structure allows multiple memory blocks mapped to the same set to coexist in the cache, thereby reducing cache misses.

[0038] In related technologies, when the computing unit reads and writes to the L1 cache, there are usually multiple data loading methods and multiple burst transmission lengths. If the cache lines are not used reasonably, the read and write efficiency of the L1 cache will be low, which will affect the execution efficiency of the computing unit in the artificial intelligence chip.

[0039] To solve the above technical problems, Figure 3 This is one of the flowcharts illustrating the cache read / write method provided by the present invention, such as... Figure 3 As shown, the method includes steps 310, 320 and 330.

[0040] Step 310: Determine the data loading method in the L1 cache for the current read / write instruction.

[0041] Specifically, the cache read / write method provided in this embodiment of the invention is executed by a cache read / write device. This device can be implemented in software, such as a cache read / write program running in an artificial intelligence chip; or it can be implemented in hardware, such as an artificial intelligence chip that executes the cache read / write method, and a computer or server containing these artificial intelligence chips.

[0042] The current read / write instruction refers to the data read / write instruction that the computing unit is currently executing. Since data may be cached in the L1 cache, the L2 cache, or memory, there are multiple data loading methods. These include loading from the L1 cache and loading from outside the L1 cache.

[0043] For read and write commands, there are usually two fields: L1 cache attribute and number of bursts.

[0044] The first-level cache attribute describes how data is loaded, and can include loading from the first-level cache (Enable L1, L1en) and loading from outside the first-level cache (Bypass L1, L1bp). Loading from the first-level cache means that the data is cached in the first-level cache. Loading from outside the first-level cache means that the data is cached only globally, bypassing the first-level cache, and is only cached in the second-level cache.

[0045] Burst transmission is a highly efficient data transmission mode. Its core feature is that it continuously transmits multiple data units in a single operation, reducing the overhead of redundant address and control signals, thereby improving system throughput and efficiency.

[0046] The burst transmission unit number refers to the number of data units transmitted consecutively in a single operation during a burst transmission. The burst transmission unit number can include burst1, burst2, and burst4. burst1 indicates that one data unit is transmitted each time, which can be 128 bytes; burst2 indicates that two consecutive data units are transmitted each time, which can be 256 bytes; and burst4 indicates that four consecutive data units are transmitted each time, which can be 512 bytes.

[0047] Step 320: Determine the target cache line based on the data loading method, metadata table, and free cache line table. The first-level cache includes multiple primary cache lines and multiple secondary cache lines; the metadata table records data tags, cache line indexes, and secondary cache line enable flags; the number of data tags corresponds to the number of primary cache lines; the free cache line table records the cache line indexes of cache lines that are currently idle.

[0048] Specifically, when loading from outside the L1 cache—that is, bypassing the L1 cache—while the computation unit is executing the current read / write instruction and waiting for the data to return, a cache line position needs to be reserved in the L1 cache to be written when the data returns. In this case, only the cache line is used, and the data tag is not required. Other read / write instructions cannot use this cache line. The number of cache lines in the L1 cache is limited; if too many are occupied, it will cause more read / write instructions to stall.

[0049] For example, in an initial state, a computing unit has two available data tags and two free cache lines in its L1 memory. When the computing unit executes a read / write instruction, the data is loaded from outside the L1 cache, requiring one cache line to be reserved for this instruction. Other read / write instructions cannot utilize the reserved cache line. When both cache lines are reserved, if the computing unit needs to execute a new read / write instruction, although two data tags are available, there is no corresponding cache line to use, causing the new read / write instruction to wait.

[0050] In this embodiment of the invention, two types of cache lines can be set. One type is the main cache line, and the other is the auxiliary cache line. The auxiliary cache line is the same size as the main cache line, for example, 256 bytes, and is used to assist in processing read and write requests executed by the computing unit, especially when the data loading method is loading from outside the L1 cache (L1bp) and the read and write instructions are burst transfer instructions (burst4).

[0051] Regarding the number of cache lines, the number of primary cache lines is equal to the number of data tags, allowing for a flexible mapping between available primary cache lines and available data tags. The number of secondary cache lines can be set according to actual needs and can be less than the number of primary cache lines; for example, the number of secondary cache lines can be one-quarter of the number of primary cache lines.

[0052] A metadata table can be configured to record data tags, cache line indexes, and secondary cache line enable flags (Useaux). The cache line indexes include the Main Cacheline Index and the Aux Cacheline Index. The secondary cache line enable flag is primarily used to determine whether secondary cache lines are enabled. The records in the metadata table are shown in Table 1.

[0053] Records in Table 1 (Meta-information Table)

[0054] Figure 4 This is a schematic diagram illustrating the mapping between data identifiers, primary cache lines, and secondary cache lines provided by the present invention, such as... Figure 4 As shown, based on the metadata table, we can obtain the mapping relationship between data identifiers and primary cache lines, whether secondary cache lines are enabled for primary cache lines, and the mapping relationship between primary cache lines and enabled secondary cache lines. The number of data identifiers is N, the number of primary cache lines is N, and the number of secondary cache lines is M. M is less than N. All of the above mapping relationships are flexible and can be determined by the metadata table.

[0055] You can also set up an idle cache line table to record the cache line index of cache lines that are in an idle state. Understandably, based on the cache line index in the idle cache line table, you can determine whether a primary or secondary cache line is in an idle state (not occupied).

[0056] The data loading method determines whether to use a primary or secondary cache line; the metadata table determines the data identifier used and the mapping relationship between the primary and secondary cache lines; the free cache line table determines which cache lines can be used; and the target cache line can be determined using these three aspects of information. The target cache line is either a cache line containing data in the first-level cache or a cache line that can cache externally loaded data.

[0057] Step 330: Based on the target cache line, obtain the data corresponding to the current read / write instruction.

[0058] Specifically, after the target cache line is determined, the externally loaded data can be stored in the target cache line for the computing unit to read and write.

[0059] The cache read / write method provided in this invention determines the data loading method of the current read / write instruction in the L1 cache; determines the target cache line based on the data loading method, a metadata table, and a free cache line table; and obtains the data corresponding to the current read / write instruction based on the target cache line. The metadata table records data tags, cache line indexes, and secondary cache line enable flags; the free cache line table records the cache line indexes of cache lines in an idle state. Since a primary cache line and a secondary cache line are set in the L1 cache, the number of data tags is equal to the number of primary cache lines, enabling flexible mapping and normal read / write operations on the L1 cache. By setting secondary cache lines, cache line resources can be flexibly supplemented under different data loading methods, allowing a flexible mapping between data tags and primary cache lines to be maintained. This reduces the mutual influence between read / write instructions loading data from outside the L1 cache and read / write instructions loading data from inside the L1 cache, improving the read / write efficiency of the L1 cache, increasing the execution efficiency of the computing unit, and enhancing the performance of the artificial intelligence chip.

[0060] It should be noted that each embodiment of the present invention can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0061] In some embodiments, determining the target cache line based on the data loading method, the metadata table, and the free cache line table includes: When the data loading method is enabled by level 1 caching, the data cache corresponding to the current read / write instruction is determined to be in the main cache line of level 1 cache; Match the data tag corresponding to the current read / write instruction with the data tag in the metadata table; The target cache line is determined based on the cache line index corresponding to the matched data tag.

[0062] Specifically, based on the current read / write instruction, the data loading method is determined to be L1 cache enabled, meaning it is loaded from the L1 cache (Enable L1, L1en). In this case, it can be determined that the data corresponding to the current read / write instruction is cached in the main cache line of the L1 cache (L1 cache hit).

[0063] The data markers corresponding to the current read / write instruction can be matched with the data markers in the metadata table. Based on the cache line index corresponding to the matching data markers in the metadata table, the hit primary cache lines are determined, and these primary cache lines are identified as target cache lines. The resulting target cache lines store the data corresponding to the current read / write instruction.

[0064] The cache read / write method provided in this embodiment of the invention can determine the target cache line based on the matching result of the data tag when the data loading mode is enabled by first-level caching, thereby improving the read / write efficiency of first-level cache.

[0065] In some embodiments, determining the target cache line based on the data loading method, the metadata table, and the free cache line table includes: When the data loading method is to bypass the first-level cache, it is determined whether the data cache corresponding to the current read / write instruction is in the second-level cache or in memory; The current secondary cache line is determined based on the free cache line table; Set the current secondary cache line as the target cache line; Remove the cache line index of the current secondary cache line from the free cache line table.

[0066] Specifically, based on the current read / write instruction, the data loading method is determined to be bypassing the L1 cache, meaning loading from outside the L1 cache (Bypass L1, L1bp). In this case, it can be determined that the data cached for the current read / write instruction is in the L2 cache or memory (L1 cache miss). While the current read / write instruction is waiting for data to return, a cache line needs to be reserved in the L1 cache, ready to be written when the data returns.

[0067] Considering that the computing unit may also execute other read and write instructions, and the data for other read and write instructions may be loaded from the L1 cache (Enable L1, L1en), in this case, the secondary cache line can be used first to reduce the use of the main cache line and reduce the impact of the current read and write instruction on other read and write instructions.

[0068] You can query the cache line index of each secondary cache line in the free cache line table. If a cache line index for a secondary cache line exists, it means that the corresponding secondary cache line is free. Select the free secondary cache line as the current secondary cache line and use it as the target cache line. The target cache line will be written with data returned from outside the first-level cache. At the same time, delete the cache line index of the current secondary cache line from the free cache line table.

[0069] After the current read / write instruction is completed, the cache line index of the current secondary cache line can be added back to the free cache line table.

[0070] The cache read / write method provided in this embodiment of the invention can prioritize the use of secondary cache lines and reduce the use of primary cache lines when the data loading method bypasses the first-level cache, thereby improving the read / write efficiency of the first-level cache.

[0071] In some embodiments, the method further includes: Based on the free cache row table, it is determined that there are no secondary cache rows in an idle state; The current main cache line is determined based on the free cache line table; Set the current main cache line as the target cache line; Remove the cache line index of the current main cache line from the free cache line table.

[0072] Specifically, if there is no cache line index for a secondary cache line in the free cache line table, it means that all secondary cache lines are occupied. In this case, the cache line index for each primary cache line can be queried from the free cache line table. If a cache line index for a primary cache line exists, it means that the current primary cache line is free. This primary cache line is then designated as the target cache line. Simultaneously, the cache line index for the current primary cache line is deleted from the free cache line table.

[0073] After the current read / write instruction is completed, the cache line index of the current main cache line can be added back to the free cache line table.

[0074] The cache read / write method provided in this embodiment of the invention only uses an idle main cache line when the idle secondary cache line has been used up, thereby improving the read / write efficiency of the first-level cache.

[0075] Each L1 cache line can store 256 bytes of data. When data needs to be read continuously from memory, burst transfer can be used, which can include burst1 (reading 128 bytes at a time), burst2 (reading 256 bytes at a time), and burst4 (reading 512 bytes at a time).

[0076] If 512 bytes need to be read using Burst 4, but this data is not in the Level 1 cache (i.e., a cache miss), the system must retrieve the data from slower external memory (such as the Level 2 cache or main memory). However, since each cache line in the Level 1 cache can only hold 256 bytes, the 512 bytes of data must be split into two parts. Therefore, the system will split this request into two Burst 2 requests (256 bytes each), storing them in two separate cache lines in the Level 1 cache. In this way, the 512 bytes of data can be read completely into the Level 1 cache.

[0077] Originally, each cache line was 256 bytes. Now, to support larger data transfers (such as burst 4), we could consider increasing the cache line size to 512 bytes. However, the total space of the L1 cache remains the same, and with larger cache lines, fewer cache lines can be stored. For example, a L1 cache with a total size of 128 kilobytes (KB) can store 512 cache lines if the cache line size is 256 bytes, but only 256 if the cache line size is 512 bytes. To keep the number of cache lines per set constant (e.g., 8-way set-associative), the total number of sets must be reduced (when changing from 256 bytes to 512 bytes, the total number of sets is halved). With fewer sets, more different data will be grouped into the same set, easily competing for space, thus reducing cache hit rate and degrading performance. Therefore, simply increasing the cache line size cannot support larger data transfer requests; otherwise, it will lead to more cache conflicts and performance degradation.

[0078] In this invention, because primary cache lines and secondary cache lines are set, the primary cache lines participate in setassociation, while the number of secondary cache lines is small and only serves as an auxiliary function. Therefore, burst4 is supported without affecting the number of primary cache lines and groups.

[0079] In some embodiments, the method further includes: The current read / write instruction is determined to be a burst transfer instruction; Determine whether the data corresponding to the current read / write instruction is cached in the L2 cache or in memory; The burst transfer length of the burst transfer instruction is determined to be greater than the maximum burst transfer length supported by the L1 cache; the maximum burst transfer length supported by the L1 cache is the data cache size of one main cache line; Based on the free cache line table, the first primary cache line and the first secondary cache line are determined; the sum of the data cache size of the first primary cache line and the first secondary cache line is equal to the burst transmission length of the burst transmission instruction; Based on the first primary cache line and the first secondary cache line, obtain the data corresponding to the current read / write instruction.

[0080] Specifically, Figure 5 This is one of the flowcharts illustrating burst transmission provided by the present invention, such as... Figure 5 As shown, when it is determined that the current read / write instruction is a burst transfer instruction, it is necessary to determine whether the L1 cache is hit, that is, whether the data of the current read / write instruction is stored in the L1 cache. If so, the data is returned directly to the computing unit; if not, it can be determined that the data corresponding to the current read / write instruction is cached in the L2 cache or memory.

[0081] The maximum burst transfer length supported by the L1 cache is the data cache size of one main cache line. For example, the maximum burst transfer length supported by the L1 cache is burst2, which is equal to the data cache size of one main cache line, or 256 bytes. However, the burst transfer length of the burst transfer instruction (the current read / write instruction) is burst4, which requires transferring 512 bytes, exceeding the maximum burst transfer length supported by the L1 cache.

[0082] The free cache line table can be queried to determine the first primary cache line and the first secondary cache line that are currently free. The sum of the data cache size of the first primary cache line and the first secondary cache line equals the burst transfer length of the burst transfer instruction. For example, if the data cache size of the first primary cache line is 256 bytes and the data cache size of the first secondary cache line is 256 bytes, the sum equals the burst transfer length of the burst transfer instruction, burst4 (512 bytes).

[0083] The system can allocate a first primary cache line and a first secondary cache line, record the corresponding data tags, the cache line index of the first primary cache line, and the cache line index of the first secondary cache line in the metadata table, and record that the secondary cache line enable flag is enabled. This establishes a mapping relationship between data tags and the first primary cache line, as well as a mapping relationship between the first primary cache line and the first secondary cache line.

[0084] You can continue to send burst transfer commands to the second-level cache or memory, and write the returned data to the first main cache line and the first secondary cache line.

[0085] The cache read / write method provided in this embodiment of the invention can use a main cache line and a secondary cache line to concatenate when the burst transmission length of the burst transmission instruction is greater than the maximum burst transmission length supported by the first-level cache. This can meet the burst transmission length requirement of burst transmission instruction burst4. In this case, since the secondary cache line is used, the set-association mode of the main cache line will not be affected, thus improving the read / write efficiency of the first-level cache.

[0086] In some embodiments, after determining that the burst transfer length of the burst transfer instruction is greater than the maximum burst transfer length supported by the L1 cache, the method further includes: Based on the free cache row table, it is determined that there are no secondary cache rows in an idle state; The current read / write instruction is split into multiple sub-read / write instructions; the burst transfer length of the sub-read / write instruction is equal to the maximum burst transfer length supported by the L1 cache; Based on the free cache line table, multiple second main cache lines are determined; the number of second main cache lines is equal to the number of sub-read / write instructions. Based on multiple secondary main cache lines, retrieve the data corresponding to the current read / write instruction.

[0087] Specifically, Figure 6 This is the second schematic diagram of the burst transmission process provided by the present invention, as follows: Figure 6 As shown, if there are no idle secondary cache lines, the current read / write instruction needs to be split into multiple sub-read / write instructions. The burst transfer length of the sub-read / write instruction is equal to the maximum burst transfer length supported by the L1 cache.

[0088] For example, the burst transfer instruction of burst4 can be split into two burst transfer instructions of burst2. burst2 is the maximum burst transfer length supported by the L1 cache.

[0089] The free cache line table can be queried to identify multiple idle secondary main cache lines. The number of secondary main cache lines is equal to the number of sub-read / write instructions. For example, if a burst transfer instruction of burst4 is split into two burst transfer instructions of burst2, and each secondary main cache line supports one burst transfer instruction of burst2, then two secondary main cache lines are required.

[0090] You can continue to send burst transfer commands to the second-level cache or memory, writing the returned data to these second-level main cache lines.

[0091] The cache read / write method provided in this embodiment of the invention, when there are no idle secondary cache lines, splits the current read / write instruction into multiple sub-read / write instructions with shorter burst transmission lengths, and realizes data transmission through multiple idle main cache lines, thereby improving the read / write efficiency of the first-level cache.

[0092] The apparatus provided in the embodiments of the present invention will be described below. The apparatus described below can be referred to in correspondence with the method described above.

[0093] Figure 7 This is a schematic diagram of the cache read / write device provided by the present invention, as shown below. Figure 7 As shown, the device includes: The loading method determination module 710 is used to determine the data loading method of the current read / write instruction in the first-level cache; The cache line determination module 720 is used to determine the target cache line based on the data loading method, the metadata table, and the free cache line table. The cache read / write module 730 is used to obtain the data corresponding to the current read / write instruction based on the target cache line; The first-level cache includes multiple primary cache lines and multiple secondary cache lines; the metadata table records data tags, cache line indexes, and secondary cache line enable identifiers; the number of data tags is equal to the number of primary cache lines; the idle cache line table records the cache line indexes of cache lines that are in an idle state.

[0094] The cache read / write device provided in this embodiment of the invention determines the data loading method of the current read / write instruction in the L1 cache; determines the target cache line based on the data loading method, a metadata table, and a free cache line table; and obtains the data corresponding to the current read / write instruction based on the target cache line. The metadata table records data tags, cache line indexes, and secondary cache line enable flags; the free cache line table records the cache line indexes of cache lines in an idle state. Since a primary cache line and a secondary cache line are set in the L1 cache, the number of data tags is equal to the number of primary cache lines, enabling flexible mapping and normal read / write operations on the L1 cache. By setting secondary cache lines, cache line resources can be flexibly supplemented under different data loading methods, allowing the data tags and primary cache lines to maintain a flexible mapping, reducing the mutual influence between read / write instructions loading data from outside the L1 cache and read / write instructions loading data from inside the L1 cache, improving the read / write efficiency of the L1 cache, improving the execution efficiency of the computing unit, and improving the performance of the artificial intelligence chip.

[0095] Figure 8 This is a schematic diagram of the structure of the artificial intelligence chip provided by the present invention, as shown below. Figure 8 As shown, the artificial intelligence chip 100 includes a streaming processor cluster 110; the streaming processor cluster 110 includes multiple computing units 111; the computing units 111 are provided with a level 1 cache 113; the computing units execute the cache read / write method in the above embodiments.

[0096] The artificial intelligence chip provided in this embodiment of the invention, because the computing unit executes the cache read and write method in the above embodiment, can make reasonable use of cache resources by utilizing the main cache line and the auxiliary cache line when loading data from outside the first-level cache or performing burst transmission, thereby improving the read and write efficiency of the first-level cache, improving the execution efficiency of the computing unit, and improving the performance of the artificial intelligence chip.

[0097] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 9 As shown, the electronic device may include: a processor 910, a communications interface 920, a memory 930, and a communications bus 940, wherein the processor 910, the communications interface 920, and the memory 930 communicate with each other via the communications bus 940. The processor 910 can call logical commands stored in the memory 930 to execute the methods described in the above embodiments, for example: The process involves determining the data loading method for the current read / write instruction in the L1 cache; identifying the target cache line based on the data loading method, the metadata table, and the free cache line table; and retrieving the data corresponding to the current read / write instruction based on the target cache line. The L1 cache includes multiple primary cache lines and multiple secondary cache lines. The metadata table records data tags, cache line indices, and secondary cache line enable flags. The number of data tags is equal to the number of primary cache lines. The free cache line table records the cache line indices of cache lines that are currently idle.

[0098] Furthermore, when the logical commands in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The processor in the electronic device provided in this embodiment of the invention can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, which will not be repeated here.

[0100] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0101] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0102] This invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cache read / write method, characterized in that, include: Determine how the current read / write instruction loads data from the L1 cache; Based on the data loading method, metadata table, and free cache line table, the target cache line is determined; Based on the target cache line, obtain the data corresponding to the current read / write instruction; The first-level cache includes multiple main cache lines and multiple secondary cache lines; the metadata table records data tags, cache line indexes, and secondary cache line enable identifiers; the number of data tags corresponds to the number of main cache lines; the idle cache line table records the cache line indexes of cache lines that are in an idle state. The method further includes: The current read / write instruction is determined to be a burst transmission instruction; It is determined that the data corresponding to the current read / write instruction is cached in the second-level cache or in memory; The burst transmission length of the burst transmission instruction is determined to be greater than the maximum burst transmission length supported by the first-level cache; the maximum burst transmission length supported by the first-level cache is the data cache size of one main cache line; Based on the free cache line table, a first primary cache line and a first secondary cache line are determined; the sum of the data cache size of the first primary cache line and the first secondary cache line is equal to the burst transmission length of the burst transmission instruction; Based on the first primary cache line and the first secondary cache line, obtain the data corresponding to the current read / write instruction.

2. The cache read / write method according to claim 1, characterized in that, The step of determining the target cache line based on the data loading method, the metadata table, and the free cache line table includes: When the data loading method is to bypass the first-level cache, it is determined that the data cached for the current read / write instruction is in the second-level cache or memory; Based on the free cache line table, determine the current secondary cache line; The current secondary cache line is determined as the target cache line; Delete the cache line index of the current secondary cache line from the free cache line table.

3. The cache read / write method according to claim 2, characterized in that, The method further includes: Based on the free cache line table, it is determined that there are no secondary cache lines in an idle state; Based on the aforementioned free cache line table, determine the current main cache line; The current main cache line is determined as the target cache line; Delete the cache line index of the current main cache line from the free cache line table.

4. The cache read / write method according to claim 1, characterized in that, The step of determining the target cache line based on the data loading method, the metadata table, and the free cache line table includes: When the data loading method is enabled by first-level caching, the data cache corresponding to the current read / write instruction is determined to be in the main cache line of the first-level cache; Match the data tag corresponding to the current read / write instruction with the data tag in the metadata table; The target cache line is determined based on the cache line index corresponding to the matched data tag.

5. The cache read / write method according to claim 1, characterized in that, After determining that the burst transmission length of the burst transmission instruction is greater than the maximum burst transmission length supported by the L1 cache, the method further includes: Based on the free cache line table, it is determined that there are no secondary cache lines in an idle state; The current read / write instruction is split into multiple sub-read / write instructions; the burst transmission length of the sub-read / write instructions is equal to the maximum burst transmission length supported by the first-level cache; Based on the free cache line table, multiple second main cache lines are determined; the number of second main cache lines is equal to the number of sub-read / write instructions. Based on the multiple second main cache lines, the data corresponding to the current read / write instruction is obtained.

6. A cache read / write device, characterized in that, include: The loading method determination module is used to determine the data loading method in the L1 cache for the current read / write instruction; The cache line determination module is used to determine the target cache line based on the data loading method, the metadata table, and the free cache line table. The cache read / write module is used to obtain the data corresponding to the current read / write instruction based on the target cache line; The first-level cache includes multiple main cache lines and multiple secondary cache lines; the metadata table records data tags, cache line indexes, and secondary cache line enable identifiers; the number of data tags corresponds to the number of main cache lines; the idle cache line table records the cache line indexes of cache lines that are in an idle state. The device is also used for: The current read / write instruction is determined to be a burst transmission instruction; It is determined that the data corresponding to the current read / write instruction is cached in the second-level cache or in memory; The burst transmission length of the burst transmission instruction is determined to be greater than the maximum burst transmission length supported by the first-level cache; the maximum burst transmission length supported by the first-level cache is the data cache size of one main cache line; Based on the free cache line table, a first primary cache line and a first secondary cache line are determined; the sum of the data cache size of the first primary cache line and the first secondary cache line is equal to the burst transmission length of the burst transmission instruction; Based on the first primary cache line and the first secondary cache line, obtain the data corresponding to the current read / write instruction.

7. An artificial intelligence chip, characterized in that, The system includes a streaming processor cluster; the streaming processor cluster includes multiple computing units; each computing unit is provided with a level-one cache; the computing unit executes the cache read / write method according to any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the cache read / write method according to any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the cache read / write method according to any one of claims 1 to 5.

Images